There are 1 GPU(s) available.
We will use the GPU: GeForce RTX 2080 Ti
------------------------------------------------------------
Loading test and validation datasets...
                                              questions  intent  ... ind  qids
0                                      customer: hello.       1  ...   2     0
1     customer: hello. agent: Hello. How may I help ...       9  ...   2     0
2     customer: hello. agent: Hello. How may I help ...       3  ...   2     0
3     customer: hello. agent: Hello. How may I help ...       4  ...   2     0
4     customer: hi. customer: i want to book a fligh...       5  ...   2     1
...                                                 ...     ...  ...  ..   ...
1103  customer: hello, i am christopher jones. agent...       6  ...   1   201
1104  customer: hello, i am christopher jones. agent...       4  ...   1   201
1105                                      customer: hi.       1  ...   1   202
1106  customer: hi. agent: Hello. How may I help you...       3  ...   1   202
1107  customer: hi. agent: Hello. How may I help you...       4  ...   1   202

[1108 rows x 5 columns]
Done!

Loading saved model from ./checkpoints/4_80.68/...
Done!

Running inference on test set...


+#1.customer: hi. customer: i want to book a flight ticket to las vegas with price under 1000. can you please help me with it?   -    7 [7] 
Actual.5 Can I know your connection limit?
P: 7 Could you please share your departure and arrival day?


+#1.customer: hi. customer: i want to book a flight ticket to las vegas with price under 1000. can you please help me with it? agent: Can I know your connection limit? customer: i need a single connection.   -    7 [7] 
Actual.15 May I know the airport codes to check?
P: 7 Could you please share your departure and arrival day?


+#2.customer: hello. agent: Hello. How may I help you? customer: can you please help me in reserving my flight from ord to reach atl? agent: Could you please share your departure and arrival day? customer: thank you, my travelling dates are apr 3rd and apr 5th.   -    5 [5] 
Actual.9 Could I have your name please?
P: 5 Can I know your connection limit?


+#2.customer: hello. agent: Hello. How may I help you? customer: can you please help me in reserving my flight from ord to reach atl? agent: Could you please share your departure and arrival day? customer: thank you, my travelling dates are apr 3rd and apr 5th. agent: Could I have your name please? customer: i am gary taylor.   -    11 [11] 
Actual.10 Do you need direct or connecting flight?
P: 11 Do you have any other specifications?


+#5.customer: hello, i am richard perez. agent: Hello. How may I help you? customer: i have an meeting at nv - las, with my company ceo. as per my company rules i need to travel only in a connecting flight.   -    7 [7] 
Actual.12 Do you have any fare limit?
P: 7 Could you please share your departure and arrival day?


-#5.customer: hello, i am richard perez. agent: Hello. How may I help you? customer: i have an meeting at nv - las, with my company ceo. as per my company rules i need to travel only in a connecting flight. agent: Do you have any fare limit? customer: my fare is 1000. agent: Could you please share your departure and arrival day? customer: my travelling dates are from 05/15 to 05/17.   -    30 [30] 
Actual.14 Can you let me know your source code?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


+#6.customer: hello. agent: Hello. How may I help you? customer: can you please help me in booking a flight ticket from denver, den to phoenix, phx? agent: Could you please share your departure and arrival day? customer: my journey dates are from aug 15 to aug 17.   -    5 [5] 
Actual.11 Do you have any other specifications?
P: 5 Can I know your connection limit?


+#8.customer: hi. agent: Hello. How may I help you? customer: i want to cancel my reservation which is done recently due to some problems, can you please help me?   -    9 [9] 
Actual.17 I will help you with cancelling the reservation.
P: 9 Could I have your name please?


+#9.customer: hi. agent: Hello. How may I help you? customer: i need to book a flight ticket from oak to ord, can you help me in booking a flight ticket? agent: Could you please share your departure and arrival day? customer: sure, my journey dates are jun 17 and jun 19.   -    5 [5] 
Actual.9 Could I have your name please?
P: 5 Can I know your connection limit?


-#9.customer: hi. agent: Hello. How may I help you? customer: i need to book a flight ticket from oak to ord, can you help me in booking a flight ticket? agent: Could you please share your departure and arrival day? customer: sure, my journey dates are jun 17 and jun 19. agent: Could I have your name please? customer: sure, rebecca moore here. agent: Do you have any other specifications? customer: no.   -    9 [9] 
Actual.30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?
P: 9 Could I have your name please?


+#12.customer: hello. agent: Hello. How may I help you? customer: can you aid me in booking a flight ticket from lga to clt? agent: Could you please share your departure and arrival day? customer: sure, i would like to go on oct 20th and return back on oct 22nd. agent: Can I know your connection limit? customer: ensure it is a single connecting flight.   -    30 [30] 
Actual.11 Do you have any other specifications?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


+#12.customer: hello. agent: Hello. How may I help you? customer: can you aid me in booking a flight ticket from lga to clt? agent: Could you please share your departure and arrival day? customer: sure, i would like to go on oct 20th and return back on oct 22nd. agent: Can I know your connection limit? customer: ensure it is a single connecting flight. agent: Do you have any other specifications? customer: ensure it is in the business class.   -    9 [9] 
Actual.8 Is there any time concern for you?
P: 9 Could I have your name please?


+#12.customer: hello. agent: Hello. How may I help you? customer: can you aid me in booking a flight ticket from lga to clt? agent: Could you please share your departure and arrival day? customer: sure, i would like to go on oct 20th and return back on oct 22nd. agent: Can I know your connection limit? customer: ensure it is a single connecting flight. agent: Do you have any other specifications? customer: ensure it is in the business class. agent: Is there any time concern for you? customer: no, i don't have any specific timings.   -    30 [30] 
Actual.11 Do you have any other specifications?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


-#12.customer: hello. agent: Hello. How may I help you? customer: can you aid me in booking a flight ticket from lga to clt? agent: Could you please share your departure and arrival day? customer: sure, i would like to go on oct 20th and return back on oct 22nd. agent: Can I know your connection limit? customer: ensure it is a single connecting flight. agent: Do you have any other specifications? customer: ensure it is in the business class. agent: Is there any time concern for you? customer: no, i don't have any specific timings. agent: Do you have any other specifications? customer: no, i don't have any other specifications.   -    30 [30] 
Actual.9 Could I have your name please?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


+#14.customer: hello. agent: Hello. How may I help you? customer: i need you to book a flight ticket from phl to bos. please help me. agent: Could you please share your departure and arrival day? customer: june 03 and 05 are my planned travelling dates.   -    5 [5] 
Actual.11 Do you have any other specifications?
P: 5 Can I know your connection limit?


+#17.customer: hi. agent: Hello. How may I help you? customer: due to the sudden demise of my granny, i am unable to catch a flight.   -    9 [9] 
Actual.31 Ok.
P: 9 Could I have your name please?


+#19.customer: hello. donald lopez here. agent: Hello. How may I help you? customer: i wan to attend a seminar in washington. can you please help me for booking the flight ticket?   -    7 [7] 
Actual.15 May I know the airport codes to check?
P: 7 Could you please share your departure and arrival day?


+#19.customer: hello. donald lopez here. agent: Hello. How may I help you? customer: i wan to attend a seminar in washington. can you please help me for booking the flight ticket? agent: May I know the airport codes to check? customer: my airport codes are from lga to dca. agent: Could you please share your departure and arrival day? customer: my travelling dates are from 09/15 to 09/17. agent: Please wait for a while. customer: make sure my connections limit is 1.   -    30 [30] 
Actual.19 I will check.
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


+#20.customer: hi. agent: Hello. How may I help you? customer: i am looking for a flight from sfo to clt, can you help me out? agent: Could you please share your departure and arrival day? customer: my trip dates are from aug 9th till aug 11th. agent: Can I know your connection limit? customer: i am looking for a connecting flight. agent: Do you have any other specifications? customer: no, thanks.   -    9 [9] 
Actual.30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?
P: 9 Could I have your name please?


+#20.customer: hi. agent: Hello. How may I help you? customer: i am looking for a flight from sfo to clt, can you help me out? agent: Could you please share your departure and arrival day? customer: my trip dates are from aug 9th till aug 11th. agent: Can I know your connection limit? customer: i am looking for a connecting flight. agent: Do you have any other specifications? customer: no, thanks. agent: Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you? customer: please proceed with it. status: book agent: Your booking has been processed successfully. customer: thank you for your help.   -    4 [4] 
Actual.9 Could I have your name please?
P: 4 Thank you for choosing us.


-#24.customer: hello, i am linda robinson. agent: Hello. How may I help you? customer: i would like to travel to sfo from oak to celebrate flowers festivals. can you please help me to book a ticket? agent: Could you please share your departure and arrival day? customer: my travelling dates will be on 01/08 to 01/10.   -    15 [15] 
Actual.11 Do you have any other specifications?
P: 15 May I know the airport codes to check?


-#24.customer: hello, i am linda robinson. agent: Hello. How may I help you? customer: i would like to travel to sfo from oak to celebrate flowers festivals. can you please help me to book a ticket? agent: Could you please share your departure and arrival day? customer: my travelling dates will be on 01/08 to 01/10. agent: Do you have any other specifications? customer: i need a connecting flight.   -    9 [9] 
Actual.30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?
P: 9 Could I have your name please?


+#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl?   -    7 [7] 
Actual.15 May I know the airport codes to check?
P: 7 Could you please share your departure and arrival day?


+#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl? agent: May I know the airport codes to check? customer: sure, my destination code is fl - mco.   -    7 [7] 
Actual.12 Do you have any fare limit?
P: 7 Could you please share your departure and arrival day?


-#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl? agent: May I know the airport codes to check? customer: sure, my destination code is fl - mco. agent: Do you have any fare limit? customer: i can afford up to 500.   -    15 [15] 
Actual.20 When are you planning to go <city>?
P: 15 May I know the airport codes to check?


-#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl? agent: May I know the airport codes to check? customer: sure, my destination code is fl - mco. agent: Do you have any fare limit? customer: i can afford up to 500. agent: When are you planning to go <city>? customer: i want to travel on 07/19.   -    30 [30] 
Actual.24 Do you need a stopover in between your route?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


-#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl? agent: May I know the airport codes to check? customer: sure, my destination code is fl - mco. agent: Do you have any fare limit? customer: i can afford up to 500. agent: When are you planning to go <city>? customer: i want to travel on 07/19. agent: Do you need a stopover in between your route? customer: yes, i need a stopover.   -    30 [30] 
Actual.22 Please help me with your return date.
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


-#25.customer: hi, i am joshua phillips. agent: Hello. How may I help you? customer: i have heard that orlando is the best place for site seeing. so, can you book a flight ticket from philadelphia, pa - phl? agent: May I know the airport codes to check? customer: sure, my destination code is fl - mco. agent: Do you have any fare limit? customer: i can afford up to 500. agent: When are you planning to go <city>? customer: i want to travel on 07/19. agent: Do you need a stopover in between your route? customer: yes, i need a stopover. agent: Please help me with your return date. customer: my arrival date is on 07/21. agent: Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you? customer: you may proceed if it is a connecting flight.   -    9 [9] 
Actual.23 Yes, it is a connecting flight.
P: 9 Could I have your name please?


+#26.customer: hello. agent: Hello. How may I help you? customer: could you please help me to book a business class ticket from los angeles to attend a business meeting in new york? agent: Could you please share your departure and arrival day? customer: i need to depart on apr 2nd and my return date will be on apr 4th.   -    5 [5] 
Actual.21 Where would you like land in <city>?
P: 5 Can I know your connection limit?


+#26.customer: hello. agent: Hello. How may I help you? customer: could you please help me to book a business class ticket from los angeles to attend a business meeting in new york? agent: Could you please share your departure and arrival day? customer: i need to depart on apr 2nd and my return date will be on apr 4th. agent: Where would you like land in <city>? customer: my boarding point is lax and my destination point is jfk.   -    11 [11] 
Actual.9 Could I have your name please?
P: 11 Do you have any other specifications?


+#26.customer: hello. agent: Hello. How may I help you? customer: could you please help me to book a business class ticket from los angeles to attend a business meeting in new york? agent: Could you please share your departure and arrival day? customer: i need to depart on apr 2nd and my return date will be on apr 4th. agent: Where would you like land in <city>? customer: my boarding point is lax and my destination point is jfk. agent: Could I have your name please? customer: i am jessica crown.   -    30 [30] 
Actual.11 Do you have any other specifications?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?


-#28.customer: hello, i am anthony turner. agent: Hello. How may I help you? customer: i would like to start my journey from chicago, il-ord to attend my family meeting at dallas, tx-dfw, can you help me in booking? agent: Could you please share your departure and arrival day? customer: i would like to start on 02/27 and my return date will be on 02/29.   -    15 [15] 
Actual.24 Do you need a stopover in between your route?
P: 15 May I know the airport codes to check?


+#29.customer: hi. i am laura walker. agent: Hello. How may I help you? customer: as i keep travelling due to the nature of my work, this time i have to travel to mco. please help me to book a flight from phl. agent: Could you please share your departure and arrival day? customer: i am planning to start my journey on nov 10th and return on nov 12th. agent: Can I know your connection limit? customer: yes, i need a connecting flight.   -    11 [11] 
Actual.8 Is there any time concern for you?
P: 11 Do you have any other specifications?


+#29.customer: hi. i am laura walker. agent: Hello. How may I help you? customer: as i keep travelling due to the nature of my work, this time i have to travel to mco. please help me to book a flight from phl. agent: Could you please share your departure and arrival day? customer: i am planning to start my journey on nov 10th and return on nov 12th. agent: Can I know your connection limit? customer: yes, i need a connecting flight. agent: Is there any time concern for you? customer: the return time should be afternoon.   -    30 [30] 
Actual.13 Do you have any class preference?
P: 30 Flight-<flight_number> is available for you with 1 connection by <aircompany> Airlines with <class_type> class with depart time at <time> hours and return time at <time> hours. Can I book it for you?
Test accuracy: 0.8068181818181818