Every simulation in the Coval org, pulled through the REST API on 2026-08-12. Completion Rate is Coval's judged verdict on whether the AI took the order to completion.
Every Completion=NO conversation (661 of them), classified by what actually happened. 489 failures (74%) are the audio-only Coval persona unable to see the order-confirm screen: the AI asks "is your order on the screen correct?", the persona asks for a readback, the AI refuses or escalates. Excluding those from numerator and denominator, completion is 91.1% against the 72.6% headline. Click any bar to see the conversations behind it, with transcript links.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
25DxMcuZszZLs94D4izC8c | Behavioral - Rush | UAT #29 | Agent did not read back the full order and escalated by saying an employee will assist. |
2UuUTSrsH3NpXjq2WyYBPN | Confused Customer | UAT #9 | Agent could not verify order alone and escalated to human for order verification. |
39q2xhhVwTd37yWw5RbMYb | Behavioral - Complex-Order | UAT #2 | Agent escalated to an employee for order verification, not completing order alone. |
3HKNLxBr55dwtpaT4rU67M | Accent - AAVE | UAT #9 | Agent said 'I can get an employee to help you verify it,' indicating escalation to a human team member. |
5qwMkniwwoBXDjFqZjbDH5 | Impatient Customer | Combo offer after sandwich | The agent repeatedly stated an employee would assist, thus escalating to a human and not completing order alone. |
6JnfFnqBvcaH6B8677qnJb | High Background Noise Speaker | UAT #61 | The agent escalated the conversation to an employee multiple times instead of directing the customer forward. |
787rPLntnU4XCHHaFYiHKM | Accent - Louisiana Creole | UAT #69 | Agent asked an employee to help with order verification, escalating to a human team member. |
7UnjnrisqzeUZeij2Gowka | High Background Noise Speaker | UAT #10 | The agent escalated to a human by stating an employee will assist the customer instead of directing them to the next win… |
8uugsoSvaqMaAxo3ko98Ya | Behavioral - Too-Talkative | UAT #10 | The agent did not finalize the order and handed off to an employee for assistance. |
9GGx8tNJEkYCfgmpogVHd6 | Confused Customer | Combo offer after signature chicken | Agent confirmed that an employee will assist, indicating escalation and not directing customer forward independently. |
9abs7kYNdqnu4XL9fLSFjY | Confused Customer | UAT #42 | The agent did not complete the order themselves and referred the customer to an employee to verify the order. |
9hKZYye4yrEdJdMXP4mLGd | Accent - Hispanic/Latino | UAT #36 | The agent escalated to a human team member instead of directing the customer to the next window. |
ADKAMQtnWSmb64cF9RtcCU | Impatient Customer | UAT #29 | Agent escalated to a human team member instead of sending customer forward after order capture. |
AMQnUUGFMG7Me3CnZ5aNFY | Accent - AAVE | Combo offer after tenders | Agent failed to read back order and escalated to a human team member for help. |
ArtfpAqrwz7JHRit3k4bbx | Low Volume Speaker | Combo offer after signature chicken | Agent did not complete order or direct customer forward; instead, agent escalated to a human employee. |
BSX6PyD4bzRqBnteziKYd2 | Confused Customer | UAT #2 | Agent escalated to a human by saying 'An employee will assist you' after order confirmation. |
CKEUUeoLkzmtkCTUvmzHrv | Confused Customer | UAT #10 | Agent escalated to an employee instead of directing to pull forward without handoff. |
CXT2vtwwmbCPGYLoHkC65V | Accent - Hispanic/Latino | UAT #8 | Agent escalated to a human team member to help verify the order before payment. |
CiLfPzLVdfbX3CtQtXyT6D | Behavioral - Rush | UAT #29 | Agent escalated to a human team member instead of directing customer to pull forward. |
DFWFmvNDYNCcv9NAEE6SqX | Behavioral - Mind-Changer | UAT #69 | Agent escalated to a human team member instead of directing customer forward after taking order. |
Showing 20 of 360.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
5E94Br2Xyvxxd79NaMEa6F | Behavioral - Complex-Order | UAT #2 | Agent failed to read back the order and told the customer to get an employee if it looked wrong. |
6gv7FeZMXry3Qd2XyoGtzn | Impatient Customer | UAT #10 | Agent failed to read back the full order and only instructed the customer to pull forward. |
74zFvHx89ADjtc24t8EbJ2 | Confused Customer | UAT #8 | The agent escalated to a human team member due to customer's difficulty with the screen and lack of readback capability. |
8SBCcweYQBnXYEfPTkzx5V | Accent - Southern US | UAT #29 | Agent escalated to a human team member to assist after user asked for order to be read back. |
9a3rqyokoB6RVFUhEEtYWb | High Background Noise Speaker | Combo offer after signature chicken | Agent failed to read back the order and escalated to a human employee for assistance. |
9p9yuQFfk27X3oGZiVPeWu | Interruptive Speaker | UAT #36 | The agent failed to read the order back and suggested a human could assist, indicating a handoff or escalation potential… |
AGMMsmZ5zeKXwjXjo6k65A | Accent - Southern US | UAT #9 | The agent escalated to a human team member to assist with reading back the order, failing to fully complete the order th… |
AwvKmeip5xUE6Ku8XYL5Ss | Confused Customer | Combo offer after sandwich | The agent escalated to a human by stating 'An employee will assist you' after the customer was confused. |
E2j3q5FEM9zoXVYHRhdQme | Low Volume Speaker | UAT #2 | Agent never read back full order and suggested an employee can help, showing incomplete handling. |
FqH3qVKCzFJPCH4BfZ3mJa | High Background Noise Speaker | UAT #9 | Agent did not direct the customer to pull forward or the next window after capturing the order. |
GM669tmg5Yb3GjGj4BFJoZ | High Background Noise Speaker | Combo offer after tenders | The agent did not direct the customer to pull forward; instead, they said 'An employee will help confirm your order,' in… |
HdbpxvPPghx23pJ7Jywfxc | Accent - Hispanic/Latino | Alternative side offering | Agent confirmed order but escalated to human when unable to read order back to customer. |
MJWQWbPQyiu5dJBUUrsDon | Behavioral - Complex-Order | Combo offer after signature chicken | Agent escalated to a human multiple times without sending customer forward or confirming order completion. |
Mh4JrSWLRRkWrj2uNeEKz8 | Accent - Hispanic/Latino | Alternative side offering | Agent escalated to a human and did not direct customer to pull forward after taking order. |
MogikpDVzrWP2SchiBQbue | Behavioral - Rush | UAT #69 | The agent escalated to a human after the customer asked for a read back of the order and could not visually confirm it. |
NK7jRHpBFGjPjic79EteJH | Standard Customer | Combo offer after tenders | Agent failed to read back the order and deferred to an employee for assistance. |
NT2u8KouYNJpqb7d9PmK37 | Accent - General American | Combo offer after sandwich | The agent said they can't read back the order and implied escalation to a human if incorrect. |
Q3c3eLvKiVkiBbeTrcM8zu | Accent - Hispanic/Latino | UAT #8 | Agent failed to read back order and escalated to an employee for assistance, not sending customer forward. |
QVtSpUf2YKVsajqZPtkVBH | Interruptive Speaker | UAT #61 | The agent did not complete the order and escalated to a human team member for assistance. |
Tw28c76BuybrZG9yjPQSwR | Behavioral - Complex-Order | UAT #69 | Agent failed to read back full order and escalated to a human before sending customer forward. |
Showing 20 of 129.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
7eWv2tggoLKZdtGYbhx5Lt | Accent - Louisiana Creole | UAT #42 | The transcript does not show the agent directing the customer to pull forward or next window after taking the full order… |
Dp7qYF8s7cHgcUJiaazeWy | Behavioral - Complex-Order | UAT #61 | Agent confirmed order but did not direct customer to pull forward or next window. |
RDpw7aeufNascPGPMWRUY3 | Impatient Customer | UAT #61 | The agent did not direct the customer to pull forward or the next window after taking the order; no instruction to proce… |
eHMYmQK9k6B7M3wW4nsTL2 | Behavioral - Mind-Changer | UAT #9 | Order was not fully captured correctly; combo size was inconsistent and agent did not clearly confirm all details before… |
75YQVG2BwKwBFdRs2BwGd5 | Behavioral - Mind-Changer | UAT #69 | The agent failed to complete the order and did not direct the customer to pull forward or the next window. |
DBGwtbfXs88tk4sYyaz2sS | Behavioral - Too-Talkative | Combo offer after sandwich | Agent asked 'Is your second side dish?' indicating the order was not fully captured before directing to the window. |
fUiciuyisjtRrP7SWSHem5 | Confused Customer | UAT #69 | The agent never captured a full order nor directed the customer to pull forward or the next window. |
gquJGMZQFT9JCGvvPSe4Pr | Super Fast Speaker | UAT #36 | The agent prematurely told the customer to pull forward before fully confirming the premium strawberry frozen lemonade a… |
Xz2jp6DSpi2EPjDps9j8NB | Behavioral - Mind-Changer | UAT #11 | Order was fully captured, but agent instructed to pull forward without confirming fries size as medium (stayed regular). |
aYjcjDnj2iBZhGVtJKYfuZ | Low Volume Speaker | Mid-order quantity change | The agent did not direct the customer to pull forward or next window after taking the order. |
kzpvRZd7P7BBWmC3iXSm3o | Accent - AAVE | UAT #64 | The agent did not direct the customer to pull forward or the next window after taking the full order. |
8SAgx7VazhuawYPjQJYZNg | Interruptive Speaker | UAT #10 | The agent confirmed the order but did not explicitly direct the customer to pull forward or proceed to the next window. |
R9xyGryrKUJjTb2DSX2DH6 | Accent - Southern US | Combo offer after signature chicken | The transcript does not show the agent directing the customer to pull forward or the next window. |
G8G6XamTqPj6QktrsNs6PH | Behavioral - Rush | UAT #29 | The order quantities and items were inconsistent and the agent did not confirm the full order accurately before telling … |
7q9nqzAFpSwUUuNfn67y2N | Behavioral - Too-Talkative | Combo offer after signature chicken | The transcript shows the order but lacks confirmation and direction to pull forward, no indication of order completion o… |
9EacVF6ozCj95ApsQg5od8 | Accent - AAVE | Combo offer after signature chicken | The agent took the order but never directed the customer to pull forward or the next window. |
A4xoyVY5yaDPA5NQtN5cuY | Low Volume Speaker | UAT #2 | The agent repeated the initial greeting and did not confirm or complete the order, nor direct the customer forward. |
CYjcTbanorT2n9f3MYRhAP | Accent - General American | UAT #2 | The agent did not confirm the full order or direct the customer to pull forward; conversation is incomplete. |
CkzJEpZNG8xpzd4ZmdZWPR | Low Volume Speaker | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window; the order was not confirmed complete. |
DZABCVo4CWd5LadwqACPHb | Accent - General American | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window in the transcript provided. |
Showing 20 of 87.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
2VnSDr8hjGzG52bK7GbNus | Accent - Hispanic/Latino | UAT #1 | The agent did not direct the customer to pull forward or the next window, instead said an employee will assist you (esca… |
5Yv8CvVdWRV2s5BvkT5mB4 | Accent - General American | UAT #1 | The agent said 'An employee will assist you,' indicating escalation to a human team member before directing to the windo… |
5YwXgKJS93LD3qDdUYQeQG | Confused Customer | Combo offer after tenders | Agent escalated to a human after customer was unsure about order confirmation. |
5czNDH2DsaV6ERMkiBFZbg | Impatient Customer | UAT #1 | Agent escalated the order by saying 'Let me get an employee to help you out.' |
6dY6NBjup6EyayNDD5T4Af | Impatient Customer | UAT #42 | Agent said, 'Let me get an employee to help you out,' indicating escalation to a human team member. |
7nU7FMtTRuZeUfgJo4HpBX | Low Volume Speaker | UAT #9 | Agent confirmed order and told customer to pull forward but then handed off to a human employee at the window. |
9uMoNS5CfY3CjB8WtgdRt4 | Confused Customer | UAT #29 | The agent handed off the conversation to a human employee instead of directing the customer to pull forward or the next … |
Av6BQ8Qm2wEyCD5cGH3yDp | Impatient Customer | UAT #64 | The agent escalated the conversation by saying 'An employee will assist you.' |
DeR3Fs93y6m6PpWnv5H2fE | Behavioral - Complex-Order | UAT #8 | Agent failed to capture customer's requested order details correctly and escalated to a larger combo with modifications. |
FoZpDqPjUDEEP52h3TdVT7 | Accent - Hispanic/Latino | UAT #1 | The agent did not complete the order or direct the customer to the next window; they escalated by saying an employee wil… |
HWxptG9gbMFgeFM9Gno6tj | Accent - AAVE | UAT #61 | Agent escalated to human by saying 'An employee will assist you' instead of sending customer forward alone. |
MfqJpvw7iDyK2jhC3FQfkD | Interruptive Speaker | UAT #8 | The agent escalated to a human team member instead of directing the customer forward without escalation. |
Nw99udeXtK5Ei4qXqXUkuX | Behavioral - Mind-Changer | UAT #10 | Agent escalated to a human employee to confirm the sandwich availability, failing to complete order independently. |
QKCb574txFhqXFdoQiMHzm | Accent - AAVE | UAT #61 | Agent captured the full order but repeatedly stated 'An employee will be right with you' instead of directing to next wi… |
VvaQ2397RQqBf7xZZD4pnC | Accent - General American | UAT #64 | Agent instructed customer to pull forward but mentioned an employee will assist, implying handoff to human. |
ajVXLjSYvZu9Fz4k2uxso4 | Accent - Hispanic/Latino | UAT #29 | Agent escalated to a human employee instead of directing customer to next window after order. |
horBgh7HjezSQB7hCCirey | Impatient Customer | Combo offer after tenders | Agent handed off to an employee instead of directing the customer forward after capturing the full order. |
ijpFLfBKPTvfZUoKPwi8cd | Low Volume Speaker | UAT #29 | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
jTR228Lcah4bspPLWu5GXQ | High Background Noise Speaker | UAT #1 | The agent repeatedly deferred to an employee for assistance and did not take the full order or direct the customer forwa… |
jXNQbfvNK4cfb8qQ6y5rnc | Standard Customer | UAT #61 | Agent did not direct customer to pull forward; repeatedly stated 'An employee will assist you,' implying escalation. |
Showing 20 of 66.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
Aa75ZNaTSzeQPT5XvCUZJp | Low Volume Speaker | UAT #29 | Agent finalized order with quantity 2 for the meal, but customer only confirmed quantity 1, indicating order not fully c… |
RkAH27zjroTHB5wQoxtEX4 | Super Fast Speaker | UAT #29 | Agent failed to capture the correct quantity of Sprites; only one was recorded instead of two, thus order not fully capt… |
Y6FLQNZdHKuEZ775jEYMC3 | Interruptive Speaker | UAT #29 | The order was not fully captured; the quantity of the 12-piece meal was only recorded as 1 instead of 2 as ordered. |
BN39pSHnBPgwCaeiWwTxsG | Accent - AAVE | UAT #2 | The agent failed to capture the full order correctly; the final confirmed order only includes the frozen lemonade, not t… |
SrNMwrjQXwPYYWBsK9L9Mi | Super Fast Speaker | UAT #29 | The order includes two Sprites but system shows only one Sprite ordered, indicating incomplete capture. |
3tKQRrLm8ybBTf9FXugAPC | Behavioral - Too-Talkative | UAT #1 | The agent ordered a Spicy Chicken Sandwich instead of the Ghost Pepper Sandwich the customer requested, so the order was… |
5rDHgtRtAjRwerx2JxdpCB | Accent - Southern US | UAT #29 | Agent confirmed order but only entered quantity 1 instead of 12-piece family meal and did not handle full quantity as pe… |
RndVPH83gz8J43MtYRPLVZ | Low Volume Speaker | UAT #2 | The agent did not capture the mango item from the customer's order, so the order is incomplete. |
7riuJnj9EsfUiUUXDHXAdH | Accent - AAVE | UAT #8 | Agent rescinded initial order choice and repeated sauces without completing the new order before sending customer forwar… |
nmtVzWDH7e4wJ3fA28HGrk | Behavioral - Too-Talkative | UAT #9 | Agent asked for a second side dish but did not update the order; order not fully captured. |
26rcRhDvDR9yBHwNwMarLC | Behavioral - Rush | UAT #2 | The order had validation issues and was not fully captured; the agent did not complete the order confirmation correctly. |
4n5v4PQKe6ziKEBXkUkvxj | Impatient Customer | UAT #2 | Agent repeated the greeting and did not capture or complete the order, nor sent customer forward. |
nTqw369hozJ6qNUGJfTPMx | Accent - Hispanic/Latino | UAT #8 | Agent confirmed order but did not address error about too many selections and proceeded to close order. |
iLof6KRxtaxRXRTvqKQhSr | Accent - Hispanic/Latino | Combo offer after signature chicken | The agent repeated the initial question mid-conversation, indicating confusion or failure to fully capture the order ini… |
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
3Q7JaCgUXJnvS38LiLtRY9 | Impatient Customer | UAT #29 | The order was not fully captured due to a discrepancy in the number of Sprites, and the agent repeated the order incorre… |
XHhkDNFudZsL2nuzwVVUa8 | Accent - General American | UAT #61 | The agent incorrectly changed the order to a dinner item, not capturing the customer's full original order. |
4kVCfEQwarxg8eDWrrSesR | Behavioral - Rush | UAT #2 | The final order summary only shows a single Classic Wrap, missing the full Wraps Bundle and lemonade, indicating incompl… |
EmXP4qEJxCx7529ece5d5N | Super Fast Speaker | UAT #29 | Agent asked about drinks after initial order but never confirmed the full order correctly (quantity mismatch) before sen… |
fzCC2drwX5WJ4mG67q66gN | Accent - Hispanic/Latino | Alternative side offering | Agent did not fully capture the order correctly and the order was inconsistent, and some items were missing. |
| Failure cause | Count | Share |
|---|---|---|
| Persona can't see screen, AI escalates | 360 | 54.5% |
| Persona can't see screen, AI refuses readback | 129 | 19.5% |
| Order fine, never directed forward | 87 | 13.2% |
| Escalated for other reasons | 66 | 10.0% |
| Other (loops, cut off, misc) | 14 | 2.1% |
| Wrong or incomplete order | 5 | 0.8% |
The widest spread in the data. Background noise and customer confusion cost more than any single scenario does, which points at the audio front end rather than the ordering logic. Persona names click through to every conversation for that persona, with verdicts and transcripts. Click a bar to see that persona's failed conversations inline.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
6JnfFnqBvcaH6B8677qnJb | UAT #61 | RBI Bakeoff | The agent escalated the conversation to an employee multiple times instead of directing the customer forward. |
7UnjnrisqzeUZeij2Gowka | UAT #10 | RBI Bakeoff | The agent escalated to a human by stating an employee will assist the customer instead of directing them to the next win… |
9a3rqyokoB6RVFUhEEtYWb | Combo offer after signature chicken | RBI Bakeoff | Agent failed to read back the order and escalated to a human employee for assistance. |
ES4AbgQErFhtU4jaUSqjnw | UAT #8 | RBI Bakeoff | The agent did not complete the order by directing the customer to pull forward; instead, they escalated to a human team … |
FqH3qVKCzFJPCH4BfZ3mJa | UAT #9 | RBI Bakeoff | Agent did not direct the customer to pull forward or the next window after capturing the order. |
GM669tmg5Yb3GjGj4BFJoZ | Combo offer after tenders | RBI Bakeoff | The agent did not direct the customer to pull forward; instead, they said 'An employee will help confirm your order,' in… |
GrHVB3hdUUfLCPqcqHzSvu | Combo offer after signature chicken | RBI Bakeoff | Agent does not send customer to pull forward or next window; no direction given after order confirmation. |
H6CEFryaPHpnMgBci2cCUS | UAT #36 | RBI Bakeoff | Agent repeatedly stated an employee would assist, indicating escalation to a human team member. |
KTnXSSmhHgm2FBn5RWda9b | UAT #36 | RBI Bakeoff | The agent escalated the conversation to a human team member instead of directing the customer to pull forward. |
Kvrczp74hr5BwtEUHT68N8 | UAT #64 | RBI Bakeoff | Agent escalated to a human employee to verify the order and assist the customer. |
LoxJ2uUjKXdLmZy4dKDPJZ | Alternative side offering | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you.' |
NWCrzEHgKjCGMAdjyjPZwN | UAT #29 | RBI Bakeoff | Agent escalated the conversation to a human employee instead of directing customer forward after full order. |
RzEBqkUfeDztAQvbw4BFau | UAT #29 | RBI Bakeoff | The agent escalated to a human team member instead of completing the order and sending the customer forward. |
ToRACtz5RohZUNmaJTP4gn | UAT #69 | RBI Bakeoff | Agent states 'I can help at the window. Employee will assist you,' indicating a handoff to a human team member. |
UVwUpUu2GC6QXwEiJEXLPS | UAT #69 | RBI Bakeoff | Agent completed order but escalated by saying 'An employee will assist you,' not directing customer to pull forward or n… |
Ve9dW96wYTHsLEeHoW7h7N | UAT #61 | RBI Bakeoff | The agent escalated the conversation to a human by stating 'An employee will assist you.' |
VpEhXmZd2WD9KMig6jCata | UAT #10 | RBI Bakeoff | Agent escalated to a human by saying an employee will assist the customer. |
WokERmgWVuSkgfXEKb7qxZ | UAT #64 | RBI Bakeoff | Agent escalated by stating 'An employee will assist you' instead of completing the order and directing customer to next … |
ZfKNoYJYMZtMeV5w4n7zPe | UAT #42 | RBI Bakeoff | The agent escalated to a human by stating an employee will assist the customer. |
ZkrjYEE4TGcxihZfidJbqg | UAT #11 | RBI Bakeoff | The agent confirmed the full order but escalated to a human employee instead of directing the customer to the next windo… |
Showing 20 of 89. Failed conversations only; the persona page has all 164.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
2UuUTSrsH3NpXjq2WyYBPN | UAT #9 | RBI Bakeoff | Agent could not verify order alone and escalated to human for order verification. |
5YwXgKJS93LD3qDdUYQeQG | Combo offer after tenders | RBI Bakeoff | Agent escalated to a human after customer was unsure about order confirmation. |
74zFvHx89ADjtc24t8EbJ2 | UAT #8 | RBI Bakeoff | The agent escalated to a human team member due to customer's difficulty with the screen and lack of readback capability. |
9GGx8tNJEkYCfgmpogVHd6 | Combo offer after signature chicken | RBI Bakeoff | Agent confirmed that an employee will assist, indicating escalation and not directing customer forward independently. |
9abs7kYNdqnu4XL9fLSFjY | UAT #42 | RBI Bakeoff | The agent did not complete the order themselves and referred the customer to an employee to verify the order. |
9uMoNS5CfY3CjB8WtgdRt4 | UAT #29 | RBI Bakeoff | The agent handed off the conversation to a human employee instead of directing the customer to pull forward or the next … |
AwvKmeip5xUE6Ku8XYL5Ss | Combo offer after sandwich | RBI Bakeoff | The agent escalated to a human by stating 'An employee will assist you' after the customer was confused. |
BSX6PyD4bzRqBnteziKYd2 | UAT #2 | RBI Bakeoff | Agent escalated to a human by saying 'An employee will assist you' after order confirmation. |
CKEUUeoLkzmtkCTUvmzHrv | UAT #10 | RBI Bakeoff | Agent escalated to an employee instead of directing to pull forward without handoff. |
LTRWCBq3wv2EgeQTFXGp3c | UAT #61 | RBI Bakeoff | Agent escalated to a human multiple times, failing to direct customer to pull forward without handoff. |
MKqfVcdheNqT3RJyw8Hd5e | UAT #61 | RBI Bakeoff | Agent did not direct customer to pull forward or the next window and repeatedly stated an employee would assist. |
PzTsXLDVHzca7EpHfXkzrX | UAT #64 | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to next window after order was captured. |
T6iatXuXt49gvSSBL3hA2k | UAT #11 | RBI Bakeoff | The agent escalated to a human employee instead of directing the customer forward after taking the full order. |
T6rgPekfobFAmsQBWCBxTd | UAT #29 | RBI Bakeoff | Agent escalated to a human employee instead of sending the customer forward alone. |
VLtEo3QDrZDNYiNFWhBkhx | Combo offer after signature chicken | RBI Bakeoff | Agent escalated to a human after failing to confirm the full order and read it back to the customer. |
VrcVPMjJrWbupJcaP7aaG5 | Mid-order quantity change | RBI Bakeoff | Agent failed to read back the full order and escalated by stating an employee would assist the customer. |
W9pZwBkwFwytgvgi4VNUws | Combo offer after tenders | RBI Bakeoff | The agent escalated to a human multiple times instead of completing order and sending customer forward. |
WuomVFj8gaUGwAxkRbDGPo | UAT #8 | RBI Bakeoff | The agent escalated to a human employee instead of sending the customer to the next window. |
Wv4RCHMWgMa8kgEBzeUoGn | Alternative side offering | RBI Bakeoff | The agent handed off the conversation to a human team member and did not direct the customer to the next window. |
XfjUww9JTAV7umTVBa9kp8 | Combo offer after sandwich | RBI Bakeoff | The agent escalated to a human when they said an employee will help and couldn't read back the order. |
Showing 20 of 91. Failed conversations only; the persona page has all 189.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
3Q7JaCgUXJnvS38LiLtRY9 | UAT #29 | RBI Bakeoff | The order was not fully captured due to a discrepancy in the number of Sprites, and the agent repeated the order incorre… |
5czNDH2DsaV6ERMkiBFZbg | UAT #1 | RBI Bakeoff | Agent escalated the order by saying 'Let me get an employee to help you out.' |
5qwMkniwwoBXDjFqZjbDH5 | Combo offer after sandwich | RBI Bakeoff | The agent repeatedly stated an employee would assist, thus escalating to a human and not completing order alone. |
6dY6NBjup6EyayNDD5T4Af | UAT #42 | RBI Bakeoff | Agent said, 'Let me get an employee to help you out,' indicating escalation to a human team member. |
6gv7FeZMXry3Qd2XyoGtzn | UAT #10 | RBI Bakeoff | Agent failed to read back the full order and only instructed the customer to pull forward. |
ADKAMQtnWSmb64cF9RtcCU | UAT #29 | RBI Bakeoff | Agent escalated to a human team member instead of sending customer forward after order capture. |
Av6BQ8Qm2wEyCD5cGH3yDp | UAT #64 | RBI Bakeoff | The agent escalated the conversation by saying 'An employee will assist you.' |
Db4jZpvxRZKvZu9t3wnHtL | UAT #9 | RBI Bakeoff | The agent escalated the order to a human team member by saying 'An employee will assist you.' |
L2ATPo3f2C2o7rQQEbfnWv | Combo offer after tenders | RBI Bakeoff | The agent said 'Employee will assist you,' indicating an escalation to a human team member. |
MAYfprdnawRdU4QxacycXn | UAT #64 | RBI Bakeoff | The agent repeatedly said an employee would assist, indicating escalation to a human team member. |
NeyqLS4uEphBsNxVgBEP3f | Combo offer after signature chicken | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member. |
RDpw7aeufNascPGPMWRUY3 | UAT #61 | RBI Bakeoff | The agent did not direct the customer to pull forward or the next window after taking the order; no instruction to proce… |
W7XH6rpEKrfUZuzEJNbEnX | UAT #36 | RBI Bakeoff | The agent escalated to a human team member multiple times instead of completing the order and directing the customer to … |
WgD6fsG3rtiC2Dz4FUDhk7 | UAT #10 | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you shortly' instead of directing the customer to pull… |
YUUfL4vEUQFLjZEqtwSLaL | UAT #8 | RBI Bakeoff | Agent failed to complete order and escalated to a human team member for assistance. |
cfXeQFyRTU9QvyXjZuPHPT | UAT #1 | RBI Bakeoff | Agent stated 'And employee will assist you,' indicating escalation to a human team member. |
eocPooQp5X99aKRv5WKYaB | UAT #42 | RBI Bakeoff | Agent escalated to a human team member twice and did not complete the order alone. |
f4YizsNNYXWxfegqC75hUH | UAT #8 | RBI Bakeoff | Agent escalated the order to a human team member instead of directing customer forward. |
fmyZAbCisqoiEm9HpfuKRD | UAT #2 | RBI Bakeoff | Agent escalated the conversation to an employee, not sending customer forward directly after full order. |
fos7hhjNTVRxLq5oyfVRwN | UAT #36 | RBI Bakeoff | Agent escalated to human after refusing to read back order despite full order capture. |
Showing 20 of 74. Failed conversations only; the persona page has all 201.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
9p9yuQFfk27X3oGZiVPeWu | UAT #36 | RBI Bakeoff | The agent failed to read the order back and suggested a human could assist, indicating a handoff or escalation potential… |
J9EZnexVDGiuAXxxVx3dWD | UAT #8 | RBI Bakeoff | The agent escalated to a human team member for order verification instead of sending the customer forward. |
MfqJpvw7iDyK2jhC3FQfkD | UAT #8 | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer forward without escalation. |
NY9Z5bszKzehraVhNd2Gct | UAT #61 | RBI Bakeoff | The agent escalated to a human employee instead of directing the customer to the next window. |
QVtSpUf2YKVsajqZPtkVBH | UAT #61 | RBI Bakeoff | The agent did not complete the order and escalated to a human team member for assistance. |
QgBHGChkg35xpCVpf4uBQw | UAT #9 | RBI Bakeoff | Agent escalated to a human team member instead of directing the customer to pull forward. |
X7VKH4QMcoNieUuKxS4Kyw | Combo offer after tenders | RBI Bakeoff | Agent escalated to a human team member by stating an employee will assist the customer further. |
Y6FLQNZdHKuEZ775jEYMC3 | UAT #29 | RBI Bakeoff | The order was not fully captured; the quantity of the 12-piece meal was only recorded as 1 instead of 2 as ordered. |
fwBtmD9RM2CPMG7UwzgPLR | Combo offer after signature chicken | RBI Bakeoff | Agent could not read back the order and directed customer to screen or employee assistance, not fully completing order t… |
nh4sNhAXFfqoBdFVbsuvCs | UAT #29 | RBI Bakeoff | Agent did not fully capture the two Sprites (only one shown) and did not explicitly direct customer to next window witho… |
6zFxG2KaR9mAxYsCQ5XV7H | Combo offer after tenders | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee instead of directing customer forward without handoff. |
PsdhafuuUnoafRTfCdQi34 | UAT #9 | RBI Bakeoff 2026-07-23 | The agent did not confirm the full order by reading it back and repeatedly handed off to a human employee for assistance… |
TvZr3txHnaQYuJR3NUxBrH | UAT #64 | RBI Bakeoff 2026-07-23 | Agent escalated to another employee to assist with order confirmation, not completing entire order themselves. |
aCohnPYtAV7LPA7jcK6ohH | UAT #61 | RBI Bakeoff 2026-07-23 | The agent modified the order to a 'Bacon & Cheese Chicken Sandwich Dinner' instead of the requested sandwich alone, indi… |
4qS8JktEibVCcXqFZrmbAd | UAT #61 | RBI Bakeoff 2026-07-24 | The agent repeatedly deferred to an employee to assist the customer, indicating escalation to a human member. |
9NMhouX5wgaxSWwMXyctgD | Combo offer after sandwich | RBI Bakeoff 2026-07-24 | Agent could not read back the order and deferred to employee assistance, indicating escalation. |
YSKSLEwi59VPstLzfcALFy | Combo offer after signature chicken | RBI Bakeoff 2026-07-24 | The agent told the customer to pull forward before the order was finalized and modified to a combo, not completing the f… |
hAVa6KL2wVSRrcWaZ4vLQN | UAT #64 | RBI Bakeoff 2026-07-24 | Agent said 'An employee will assist you,' indicating handoff to a human team member before directing customer forward. |
jX3JoEcM72iGmjnrYxuFfA | UAT #9 | RBI Bakeoff 2026-07-24 | Agent escalated to a human employee to assist the customer instead of directing them to pull forward. |
noW5MKjKmKQeYCypkFxhm5 | Combo offer after tenders | RBI Bakeoff 2026-07-24 | The agent did not direct the customer to pull forward/next window and escalated to a human assistant instead. |
Showing 20 of 34. Failed conversations only; the persona page has all 134.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
39q2xhhVwTd37yWw5RbMYb | UAT #2 | RBI Bakeoff | Agent escalated to an employee for order verification, not completing order alone. |
5E94Br2Xyvxxd79NaMEa6F | UAT #2 | RBI Bakeoff | Agent failed to read back the order and told the customer to get an employee if it looked wrong. |
DFpYUqgufGpz9Q9R7tZJcw | Combo offer after sandwich | RBI Bakeoff | Agent escalated to a human employee instead of directing customer forward. |
DeR3Fs93y6m6PpWnv5H2fE | UAT #8 | RBI Bakeoff | Agent failed to capture customer's requested order details correctly and escalated to a larger combo with modifications. |
Dp7qYF8s7cHgcUJiaazeWy | UAT #61 | RBI Bakeoff | Agent confirmed order but did not direct customer to pull forward or next window. |
MJWQWbPQyiu5dJBUUrsDon | Combo offer after signature chicken | RBI Bakeoff | Agent escalated to a human multiple times without sending customer forward or confirming order completion. |
Tw28c76BuybrZG9yjPQSwR | UAT #69 | RBI Bakeoff | Agent failed to read back full order and escalated to a human before sending customer forward. |
ZJtDSjGcMKfm2AnecCopvb | UAT #10 | RBI Bakeoff | The agent escalated the order to a human employee for assistance instead of completing it themselves. |
cScsqL2sK7pZpLHftzspCU | Combo offer after tenders | RBI Bakeoff | Agent escalated to a human team member upon customer's request to read back the order. |
oPgA5FxqrBgjfjEUJih5B2 | UAT #10 | RBI Bakeoff | Agent handed off to a human team member instead of directing customer to the next window. |
KEAk2eeF52K4P7aj6wznmr | Combo offer after signature chicken | RBI Bakeoff 2026-07-23 | Agent repeatedly mentioned 'An employee will assist you' indicating escalation to a human. |
KuurwUdQaQ63XpMEpksHnP | UAT #2 | RBI Bakeoff 2026-07-23 | The agent did not complete the order but repeatedly escalated to a human employee to assist. |
Y84TaPT6kY2iKPCmuBs5my | UAT #29 | RBI Bakeoff 2026-07-23 | The agent did not read back the full order and repeatedly escalated to a human employee to assist the customer. |
YXsRTxMJKfvhPDrtfbmTan | Alternative side offering | RBI Bakeoff 2026-07-23 | The agent failed to read back the order and escalated to a human employee for assistance. |
ZbJT8HG6tAG5JSpKGMEhz2 | UAT #64 | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member stating 'An employee will assist you'. |
fxzKtjGtxUKXCYngNZLnhD | UAT #9 | RBI Bakeoff 2026-07-23 | Agent did not read back the full order and escalated to a human employee for verification. |
mmMGRYoQ7AaBLW55do3Ja9 | UAT #69 | RBI Bakeoff 2026-07-23 | Agent failed to read back the order and directed customer to pull forward for employee assistance. |
2ws4Q5CkdeJveK2XSjWVRy | UAT #61 | RBI Bakeoff 2026-07-24 | The agent did not read back the order and mentioned an employee would assist if needed, indicating a handoff or escalati… |
6SidWtNFX9dppnNt9HqXEU | UAT #10 | RBI Bakeoff 2026-07-24 | The agent could not read back the order and said an employee will assist the customer, indicating escalation to a human. |
CEQ3nXsXs8jgaSSYaHT6Vf | UAT #29 | RBI Bakeoff 2026-07-24 | Agent failed to confirm the full two-piece order correctly and denied reading back, implying incomplete capture and pote… |
Showing 20 of 51. Failed conversations only; the persona page has all 209.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
7nU7FMtTRuZeUfgJo4HpBX | UAT #9 | RBI Bakeoff | Agent confirmed order and told customer to pull forward but then handed off to a human employee at the window. |
Aa75ZNaTSzeQPT5XvCUZJp | UAT #29 | RBI Bakeoff | Agent finalized order with quantity 2 for the meal, but customer only confirmed quantity 1, indicating order not fully c… |
ArtfpAqrwz7JHRit3k4bbx | Combo offer after signature chicken | RBI Bakeoff | Agent did not complete order or direct customer forward; instead, agent escalated to a human employee. |
DL5ywZJmjcy29M5bSjw2em | Combo offer after tenders | RBI Bakeoff | The agent ended up saying 'An employee will assist you,' indicating escalation to a human. |
E2j3q5FEM9zoXVYHRhdQme | UAT #2 | RBI Bakeoff | Agent never read back full order and suggested an employee can help, showing incomplete handling. |
Lh9ua2hG87bwcbBK4SrmqB | UAT #1 | RBI Bakeoff | Agent did not complete the order and escalated to a human employee for assistance. |
ULHkYRxYbQMDoxewqMs7n3 | UAT #69 | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer to pull forward. |
c9kAN9NR2xRBAq7J9ayPam | UAT #61 | RBI Bakeoff | Agent escalated the order by saying an employee will assist the customer. |
gXqxN9aKm8cFfCSZvk7wiU | UAT #69 | RBI Bakeoff | The agent did not direct the customer to pull forward/next window and instead escalated to a human employee. |
ijpFLfBKPTvfZUoKPwi8cd | UAT #29 | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
mUzsUGP5LC3fU4BsCEy2jM | Combo offer after signature chicken | RBI Bakeoff | Agent escalated to a human employee for assistance instead of directing customer to pull forward. |
ngSqCHrPPi5xSAbutuPhf6 | UAT #11 | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member. |
BCYd2taWD9nPWtxYyGhcPq | UAT #1 | RBI Bakeoff 2026-07-23 | Agent escalated to a human after taking the order, not sending customer forward directly. |
SLbhtvXmVSpvHXPqkUogRk | UAT #9 | RBI Bakeoff 2026-07-23 | The agent escalated to a human employee when the order was not confirmed correctly. |
bfsknDbidbtLyFnMTpqQpQ | Mid-order quantity change | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member, saying 'An employee will assist you.' |
ibkXkkQFhqfiY7sE5Pp7Tt | UAT #69 | RBI Bakeoff 2026-07-23 | Agent escalated to a human to verify the order, not directing customer forward themselves. |
JXnEp9qujCbh6geQjWq9V4 | Combo offer after sandwich | RBI Bakeoff 2026-07-24 | Agent did not complete the order; instead, said an employee will assist, indicating escalation. |
LfVRoQcdeevrWsbDK6z52Q | UAT #61 | RBI Bakeoff 2026-07-24 | Agent escalated the order by stating an employee will assist, not directing customer to pull forward. |
RndVPH83gz8J43MtYRPLVZ | UAT #2 | RBI Bakeoff 2026-07-24 | The agent did not capture the mango item from the customer's order, so the order is incomplete. |
ZY95QrtVwy7yC6qwNRd72n | Combo offer after signature chicken | RBI Bakeoff 2026-07-24 | The agent said 'An employee will help confirm it,' indicating escalation to a human team member. |
Showing 20 of 35. Failed conversations only; the persona page has all 147.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
25DxMcuZszZLs94D4izC8c | UAT #29 | RBI Bakeoff | Agent did not read back the full order and escalated by saying an employee will assist. |
CiLfPzLVdfbX3CtQtXyT6D | UAT #29 | RBI Bakeoff | Agent escalated to a human team member instead of directing customer to pull forward. |
MogikpDVzrWP2SchiBQbue | UAT #69 | RBI Bakeoff | The agent escalated to a human after the customer asked for a read back of the order and could not visually confirm it. |
Qhboofo7T9Fx9C64Ef5sYG | UAT #36 | RBI Bakeoff | Agent escalated to a human team member by saying an employee will assist the customer. |
WXafFYP4LYmodicsJHgCTU | UAT #2 | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you.' |
ey3T5vRhd4T88qubuAmvxG | UAT #10 | RBI Bakeoff | Agent escalated to a human team member instead of sending the customer forward after order capture. |
f9op3UXPtESqFWfPDtYRzm | Alternative side offering | RBI Bakeoff | The agent said 'No problem. An employee will assist you,' indicating a handoff to a human team member. |
jgGBpyZih8JDxLyFaWXxpy | Combo offer after signature chicken | RBI Bakeoff | The agent stated 'An employee will assist you,' indicating escalation and not completing the order interaction fully alo… |
nstFbStcCebtsttZgBAmeL | Alternative side offering | RBI Bakeoff | Agent escalated to a human employee instead of directing customer forward without handoff. |
DzPQcP7FUKjdcZJVWEA2mD | UAT #11 | RBI Bakeoff 2026-07-23 | The agent said an employee will assist, indicating escalation to a human team member. |
JTPGTEfRKLCEDMT9fjxU3C | UAT #9 | RBI Bakeoff 2026-07-23 | The agent escalated the order verification to a human employee and did not send customer forward without escalation. |
NW4LNmdM6Sc3ePmfu5DZyp | Alternative side offering | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member for order verification instead of directing customer forward. |
V5XnWTFfqeCtrm4sjiL9Jj | UAT #29 | RBI Bakeoff 2026-07-23 | Agent repeatedly told customer an employee will assist and never confirmed complete order or directed to pull forward. |
is9D6ybbKXpJrv6fJTkFYj | UAT #1 | RBI Bakeoff 2026-07-23 | Agent escalated to human assistance for Ghost Pepper sandwich issue. |
26aQqRjiNVAa9fLmcBSQv6 | UAT #10 | RBI Bakeoff 2026-07-24 | Agent escalated to a human to help confirm the order, failing to complete it fully. |
RAyEvRBHYXovj7iyL4Saaq | UAT #36 | RBI Bakeoff 2026-07-24 | Agent repeatedly escalated to a human employee instead of directing customer forward without handoff. |
hCFZN8xJPrCmWrqTqKjgtQ | Combo offer after tenders | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member instead of directing customer to pull forward. |
mQDmShsdY22C3YsduuuYXq | UAT #29 | RBI Bakeoff 2026-07-24 | Agent escalated to a human multiple times and did not direct customer to pull forward directly. |
njJks7FYBbgpEDbTimGrvm | UAT #69 | RBI Bakeoff 2026-07-24 | The agent escalated to a human to verify the order instead of directing the customer forward independently. |
2K2Go7mxi3nnp9cxjCZYRb | Combo offer after tenders | RBI Bakeoff 2026-07-24 Pass 5 | The agent could not read back the order and mentioned an employee would assist if it looked wrong, indicating escalation… |
Showing 20 of 39. Failed conversations only; the persona page has all 175.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
EsFDbbGBBZYkEk7n3sgfuq | UAT #69 | RBI Bakeoff | Agent escalated to a human employee instead of completing the order and directing the customer forward. |
FXFj9jg6YsovEeRmiNVFb4 | UAT #9 | RBI Bakeoff | Agent escalated to a human after taking order and didn't direct customer forward themselves. |
NK7jRHpBFGjPjic79EteJH | Combo offer after tenders | RBI Bakeoff | Agent failed to read back the order and deferred to an employee for assistance. |
SsLiSbEL8yUhekVkjwMFd9 | UAT #2 | RBI Bakeoff | The agent escalated the order confirmation to a human employee multiple times without finalizing the order or directing … |
YGgSbDVwNtRxXsLaFMEa95 | UAT #61 | RBI Bakeoff | The agent escalated to a human employee and did not complete the order themselves. |
ccTAHm7KPQxA9TkErBzCQF | UAT #9 | RBI Bakeoff | The agent escalated to a human team member multiple times instead of sending customer forward without handoff. |
duJRLxY6i7YYjYkRNbUPDQ | UAT #42 | RBI Bakeoff | The agent escalated to a human team member instead of completing the order and sending the customer forward. |
jW5sj4gJVN9RecmoVkFPCz | UAT #29 | RBI Bakeoff | The order was not fully captured correctly (Sprite quantity was inconsistent), so full order was not confirmed before as… |
jXNQbfvNK4cfb8qQ6y5rnc | UAT #61 | RBI Bakeoff | Agent did not direct customer to pull forward; repeatedly stated 'An employee will assist you,' implying escalation. |
FwQcA6knF5XDZEFZM4LYqZ | Combo offer after tenders | RBI Bakeoff 2026-07-23 | Agent failed to fully read back the order and deferred to employee assistance, indicating incomplete order capture. |
XdxoMyu8hgJEMy4azMGTBX | UAT #29 | RBI Bakeoff 2026-07-23 | The order was not fully captured as the agent only confirmed one Sprite instead of two. |
eaqaMbvw8fcFS3y9NDj6X4 | UAT #61 | RBI Bakeoff 2026-07-23 | Agent repeatedly says an employee will assist instead of directing the customer to pull forward or next window. |
gJxbRi8QVVQCm4Zidmc8GA | UAT #9 | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member twice instead of directing customer forward after taking full order. |
2fwqYYbv6kNoNoxQ6y8QbT | UAT #61 | RBI Bakeoff 2026-07-24 | Agent confirmed order but escalated to a human instead of directing customer to next window. |
HLMmf53ijZkjraDDq6HhB5 | UAT #29 | RBI Bakeoff 2026-07-24 | The agent did not fully read back the order and escalated to a human team member instead of sending the customer forward… |
KNCcRCfSxNog7XbrLfy335 | UAT #9 | RBI Bakeoff 2026-07-24 | Agent escalated to a human after the customer requested to hear the order back. |
TKDtcEnLqjtqRAFtX8VtPF | UAT #36 | RBI Bakeoff 2026-07-24 | The agent did not read back the full order and deferred to a human employee for verification multiple times. |
njmQqZ7WFcsKLw6Kwu7JJZ | Combo offer after sandwich | RBI Bakeoff 2026-07-24 | Agent failed to read back order and escalated to a human employee for assistance. |
oQStC82y3waiiCcVg3JQZ9 | UAT #2 | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member multiple times and did not direct customer to pull forward/next window. |
Gvo8rCud6DoLTtLkpG7hVt | UAT #36 | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human team member instead of confirming the full order and directing customer forward. |
Showing 20 of 41. Failed conversations only; the persona page has all 183.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
5Yv8CvVdWRV2s5BvkT5mB4 | UAT #1 | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member before directing to the windo… |
DoXBp5QsXvE3gt6m9xQrjA | Combo offer after signature chicken | RBI Bakeoff | The agent escalated to a human employee for assistance, not sending customer forward without escalation. |
FVxknVXBN86xwBFyNHi6a3 | UAT #9 | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward after capturing order. |
HFN3zo9apgbMw9mBELF3Ns | Combo offer after signature chicken | RBI Bakeoff | Agent escalated the order to a human team member for verification instead of directing the customer forward. |
NT2u8KouYNJpqb7d9PmK37 | Combo offer after sandwich | RBI Bakeoff | The agent said they can't read back the order and implied escalation to a human if incorrect. |
VvaQ2397RQqBf7xZZD4pnC | UAT #64 | RBI Bakeoff | Agent instructed customer to pull forward but mentioned an employee will assist, implying handoff to human. |
XHhkDNFudZsL2nuzwVVUa8 | UAT #61 | RBI Bakeoff | The agent incorrectly changed the order to a dinner item, not capturing the customer's full original order. |
HDmSAjKSmR6wZXetPgicT3 | UAT #9 | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member to read back the order, not directing customer to next window. |
Q98gcqqkbqGpDCHTwJYeEv | UAT #11 | RBI Bakeoff 2026-07-23 | The agent did not read back the order and escalated to a human team member for assistance. |
6ayoJPdJdWkUi6kxwk78Gy | UAT #36 | RBI Bakeoff 2026-07-24 | Agent did not read back order and referred customer to an employee, indicating escalation. |
CpkZKkPQeYpTN3GSUKoZQK | UAT #1 | RBI Bakeoff 2026-07-24 | The agent escalated to a human employee multiple times to assist with order verification. |
TAa2nXRC7YufCbNWMoRJUm | Combo offer after tenders | RBI Bakeoff 2026-07-24 | The agent escalated to a human team member for assistance twice instead of directing the customer forward. |
kwANnpoEfzEP2zCcqbztNh | UAT #61 | RBI Bakeoff 2026-07-24 | Agent could not read the order back and implied human assistance might be needed. |
ED9tqbqhXYGcFNGYqBwQkJ | Combo offer after tenders | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human employee for order verification instead of directing customer forward. |
JLexbjZxHm9qgaGnJy994Q | Mid-order quantity change | Simulation Run 2026-07-20 23:03 (scratch) | Agent repeatedly said 'An employee will assist you,' indicating escalation to a human. |
gvQHmw3XQoFU2ZnkjSEoNC | Combo offer after sandwich | Simulation Run 2026-07-20 23:03 (scratch) | The agent escalated to a human team member by saying 'An employee will assist you.' |
CYjcTbanorT2n9f3MYRhAP | UAT #2 | Simulation Run 2026-08-16 21:10 | The agent did not confirm the full order or direct the customer to pull forward; conversation is incomplete. |
DZABCVo4CWd5LadwqACPHb | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The agent did not direct the customer to pull forward or the next window in the transcript provided. |
gxPsR3qibBMyBivqrym3FB | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The transcript contains no agent confirmation or direction to pull forward/next window, only the customer's repeated ord… |
3a2gaMEbNEh9aZAh9FeFPY | UAT #9 | Simulation Run 2026-08-17 01:07 | The agent did not complete the order or direct the customer to pull forward or the next window. |
Showing 20 of 25. Failed conversations only; the persona page has all 145.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
2VnSDr8hjGzG52bK7GbNus | UAT #1 | RBI Bakeoff | The agent did not direct the customer to pull forward or the next window, instead said an employee will assist you (esca… |
9hKZYye4yrEdJdMXP4mLGd | UAT #36 | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer to the next window. |
CXT2vtwwmbCPGYLoHkC65V | UAT #8 | RBI Bakeoff | Agent escalated to a human team member to help verify the order before payment. |
EA8kCDWmqYTZSEkHCB9CkU | UAT #9 | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
FoZpDqPjUDEEP52h3TdVT7 | UAT #1 | RBI Bakeoff | The agent did not complete the order or direct the customer to the next window; they escalated by saying an employee wil… |
HdbpxvPPghx23pJ7Jywfxc | Alternative side offering | RBI Bakeoff | Agent confirmed order but escalated to human when unable to read order back to customer. |
LF3uBCaLVSprakLXAEaZbS | UAT #2 | RBI Bakeoff | The agent escalated to a human by saying 'I'll get an employee to help you.' |
Lp9GjC3s76QDdHMvMp6B2j | Mid-order quantity change | RBI Bakeoff | The agent escalated to a human team member by stating an employee would assist the customer. |
Mh4JrSWLRRkWrj2uNeEKz8 | Alternative side offering | RBI Bakeoff | Agent escalated to a human and did not direct customer to pull forward after taking order. |
Q3c3eLvKiVkiBbeTrcM8zu | UAT #8 | RBI Bakeoff | Agent failed to read back order and escalated to an employee for assistance, not sending customer forward. |
QgYXeANSjaxZdN4oEurE8T | UAT #10 | RBI Bakeoff | Agent escalated to a human team member by saying 'An employee will assist you.' |
QmL3kths2eNuEQLtWyrr5s | UAT #29 | RBI Bakeoff | The agent said 'An employee will assist you,' indicating an escalation to a human team member. |
RMz8UL6KWxRKm5wFa7YqBv | UAT #10 | RBI Bakeoff | Agent escalated to a human team member by stating an employee will assist the customer repeatedly. |
YjJP6rfLhzTCmqkxwh7jui | Combo offer after tenders | RBI Bakeoff | The agent said 'Employee will assist you,' indicating escalation to a human team member. |
ZBLBgDuuU9cgiX38DcLGsJ | Combo offer after tenders | RBI Bakeoff | Agent escalated to a human by saying an employee will assist the customer. |
ajVXLjSYvZu9Fz4k2uxso4 | UAT #29 | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to next window after order. |
kc6zGK3sugeqJWgcy56ijw | UAT #9 | RBI Bakeoff | The agent escalated to a human employee for order confirmation multiple times instead of directing the customer to the n… |
5iZvibFE8hdVKSEAJS75Kh | UAT #8 | RBI Bakeoff 2026-07-23 | Agent escalated the order to a human team member instead of sending the customer forward themselves. |
JWpHuC8G9f7gYG7y3Ct7vt | UAT #61 | RBI Bakeoff 2026-07-23 | Agent escalated to an employee instead of directing customer to next window. |
P8bSLwf2ySSbvmrpPUgfNe | UAT #10 | RBI Bakeoff 2026-07-23 | The agent escalated the order to a human employee instead of directing the customer to pull forward. The order was hande… |
Showing 20 of 69. Failed conversations only; the persona page has all 384.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
3HKNLxBr55dwtpaT4rU67M | UAT #9 | RBI Bakeoff | Agent said 'I can get an employee to help you verify it,' indicating escalation to a human team member. |
AMQnUUGFMG7Me3CnZ5aNFY | Combo offer after tenders | RBI Bakeoff | Agent failed to read back order and escalated to a human team member for help. |
HWxptG9gbMFgeFM9Gno6tj | UAT #61 | RBI Bakeoff | Agent escalated to human by saying 'An employee will assist you' instead of sending customer forward alone. |
QKCb574txFhqXFdoQiMHzm | UAT #61 | RBI Bakeoff | Agent captured the full order but repeatedly stated 'An employee will be right with you' instead of directing to next wi… |
U9QMYnopq8EUmXt7vmzg9f | UAT #42 | RBI Bakeoff | Agent could not read back order and deferred to employee assistance, indicating escalation or bail-out. |
dLrXgUa8M29ZC5uiTaw82f | UAT #11 | RBI Bakeoff | The agent escalated to a human team member rather than directing the customer to pull forward. |
hCFCxjw29pgjoZqzeTP3Tc | Combo offer after tenders | RBI Bakeoff | Agent changed the order from 5 classic tenders to a 3 Piece Chicken Tenders Dinner without clarifying; order incomplete. |
oAjPgFzupR3Z7xhGEemVYk | Combo offer after sandwich | RBI Bakeoff | Agent said 'Employee will assist you,' indicating escalation to a human team member. |
BN39pSHnBPgwCaeiWwTxsG | UAT #2 | RBI Bakeoff 2026-07-23 | The agent failed to capture the full order correctly; the final confirmed order only includes the frozen lemonade, not t… |
XnZc9FQ7uskE5tGe7pSABh | UAT #61 | RBI Bakeoff 2026-07-23 | Agent did not direct the customer to pull forward; instead, they said an employee would assist, indicating escalation. |
Qbx9ThzRXK7e72qyX4bW9p | UAT #42 | RBI Bakeoff 2026-07-24 | Agent did not read back the full order and escalated to a human for assistance. |
aNLtZaBRr945p6z9JZWAji | UAT #9 | RBI Bakeoff 2026-07-24 | The agent repeatedly refused to read back the order and escalated to an employee for assistance. |
dDezyevYC5oDSBVHxQu6xi | Alternative side offering | RBI Bakeoff 2026-07-24 | The agent escalated to a human by saying 'An employee will assist you,' not sending the customer forward directly. |
7riuJnj9EsfUiUUXDHXAdH | UAT #8 | RBI Bakeoff 2026-07-24 Pass 5 | Agent rescinded initial order choice and repeated sauces without completing the new order before sending customer forwar… |
RMnyAs38BCaaPBPR9ZXEqL | Combo offer after tenders | RBI Bakeoff 2026-07-24 Pass 5 | The agent failed to fully capture the correct order and indicated an employee would assist, showing incomplete order han… |
kzpvRZd7P7BBWmC3iXSm3o | UAT #64 | RBI Bakeoff 2026-07-24 Pass 5 | The agent did not direct the customer to pull forward or the next window after taking the full order. |
2WL4Gc83b4brtEJJPt7Re7 | UAT #11 | Simulation Run 2026-07-20 23:03 (scratch) | The agent said 'Employee will help fix that,' indicating escalation to a human. |
XZnRCCHeEMzeQao5QzXYAX | Combo offer after sandwich | Simulation Run 2026-07-20 23:03 (scratch) | Agent did not direct the customer to pull forward/next window but repeatedly escalated to an employee instead. |
g8TaRYP8Zy8ZFusKMm4UEA | UAT #9 | Simulation Run 2026-07-20 23:03 (scratch) | Agent escalated to a human multiple times and did not confirm order or direct to next window alone. |
9EacVF6ozCj95ApsQg5od8 | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The agent took the order but never directed the customer to pull forward or the next window. |
Showing 20 of 27. Failed conversations only; the persona page has all 156.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
DFWFmvNDYNCcv9NAEE6SqX | UAT #69 | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward after taking order. |
Nw99udeXtK5Ei4qXqXUkuX | UAT #10 | RBI Bakeoff | Agent escalated to a human employee to confirm the sandwich availability, failing to complete order independently. |
SKmu5fyiSiEgMugKqCvCht | UAT #8 | RBI Bakeoff | Agent escalated to a human team member instead of directing customer to pull forward without escalation. |
aMLQVa8AV2jhvcsRJNZJDr | Combo offer after sandwich | RBI Bakeoff | The agent initially told the customer to pull forward before the order was complete, and the customer changed the order … |
eHMYmQK9k6B7M3wW4nsTL2 | UAT #9 | RBI Bakeoff | Order was not fully captured correctly; combo size was inconsistent and agent did not clearly confirm all details before… |
75YQVG2BwKwBFdRs2BwGd5 | UAT #69 | RBI Bakeoff 2026-07-23 | The agent failed to complete the order and did not direct the customer to pull forward or the next window. |
7nwTK5uxnM3muZmRZvbdsp | Combo offer after sandwich | RBI Bakeoff 2026-07-23 | Agent escalated to a human by saying 'An employee will assist you' after taking the full order. |
NrUhuZ4jqkBsHCxiSwvGA5 | UAT #64 | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member for order verification instead of directing the customer forward. |
SgK4m572F7oKUKcjcQgXUL | UAT #10 | RBI Bakeoff 2026-07-23 | The agent repeatedly said 'An employee will assist you' instead of directing the customer to pull forward or the next wi… |
cffXevwixHsDxQvBATUAP5 | Combo offer after tenders | RBI Bakeoff 2026-07-23 | The agent did not complete the order and escalated to a human employee for assistance. |
44XzyUDLxvjK3MgRrdzQQa | UAT #29 | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member multiple times and did not send the customer forward without escalation. |
4ZYPMs7Y2EKcT3qXDmuo7Q | UAT #42 | RBI Bakeoff 2026-07-24 | Agent couldn't read back the full order and suggested employee help, indicating incomplete autonomous handling. |
KNNgE8K8afxMQRHU8VzDf5 | UAT #2 | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member at the window for order review, not completing order alone. |
JWMN2ppfHyyp43GNFQ4JJQ | Alternative side offering | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human by saying an employee will assist them without completing the order fully themselves. |
Xz2jp6DSpi2EPjDps9j8NB | UAT #11 | RBI Bakeoff 2026-07-24 Pass 5 | Order was fully captured, but agent instructed to pull forward without confirming fries size as medium (stayed regular). |
Z6b9LshTLUvn6F8efa7daw | UAT #8 | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human by saying an employee will assist, thus not completing the order fully themselves. |
ahU9omddbQ52qRR5SspmKy | UAT #69 | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to an employee for order verification instead of completing the order directly. |
52B7YGvu3r5CoRnRQNn7X2 | Mid-order quantity change | Simulation Run 2026-07-20 23:58 (scratch) | Agent confirmed order but then said 'An employee will assist you,' indicating escalation to a human. |
GAAadyc8EMYMBm2tx8Eq7L | UAT #8 | Simulation Run 2026-07-20 23:58 (scratch) | Agent failed to confirm the final order and escalated to a human employee for assistance. |
SWwgoj5WiAfxPExPhJzGHL | UAT #69 | Simulation Run 2026-07-20 23:58 (scratch) | Agent escalated to a human employee instead of directing the customer to the next window after taking the full order. |
Showing 20 of 23. Failed conversations only; the persona page has all 160.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
RkAH27zjroTHB5wQoxtEX4 | UAT #29 | RBI Bakeoff | Agent failed to capture the correct quantity of Sprites; only one was recorded instead of two, thus order not fully capt… |
SEcqstDg8dGQTe2uYBfUFV | UAT #2 | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to pull forward/next window after capturing order. |
WobG2LCx386BoD5DtnEMWb | Combo offer after tenders | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward without escalation. |
QqQcW5hMJataJfZfyVFyor | Combo offer after tenders | RBI Bakeoff 2026-07-23 | The agent said 'An employee will assist you,' indicating escalation to a human. |
SrNMwrjQXwPYYWBsK9L9Mi | UAT #29 | RBI Bakeoff 2026-07-23 | The order includes two Sprites but system shows only one Sprite ordered, indicating incomplete capture. |
VRuFNWaLkFFee3EfqqR83R | UAT #42 | RBI Bakeoff 2026-07-23 | The agent escalated to a human employee to verify the order instead of completing it independently. |
WbEXyeKLfNMebSr3pNuTFZ | UAT #61 | RBI Bakeoff 2026-07-23 | Agent escalated by stating "An employee will assist you," not directing customer to pull forward/next window. |
eUvrpzDVVhtVhrAVFrhkps | UAT #36 | RBI Bakeoff 2026-07-23 | The agent escalated to a human by saying an employee will assist the customer. |
gquJGMZQFT9JCGvvPSe4Pr | UAT #36 | RBI Bakeoff 2026-07-24 | The agent prematurely told the customer to pull forward before fully confirming the premium strawberry frozen lemonade a… |
9a4D9BBy4Bcg7XP9zdy5zR | UAT #2 | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated the conversation to an employee instead of directing the customer to pull forward. |
ngas8HE8wkM77FYmpH9qg9 | Combo offer after tenders | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human team member instead of directing customer forward. |
EmXP4qEJxCx7529ece5d5N | UAT #29 | Simulation Run 2026-07-20 22:35 (scratch) | Agent asked about drinks after initial order but never confirmed the full order correctly (quantity mismatch) before sen… |
ivqYqMRoGU2FsyTm6oRrP6 | Alternative side offering | Simulation Run 2026-07-20 22:35 (scratch) | The agent escalated to a human team member by saying 'An employee will assist you.' |
SEJ3Zib2HNrghQNaJeHxma | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | Agent escalated the order to a large combo, modifying original item, indicating partial capture and shift. |
UhSitR9h3mJKLTANURCjog | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The agent escalated to a human team member instead of completing the order and directing forward. |
df2PcMXnYcHpypDV4r6bYY | UAT #61 | Simulation Run 2026-08-17 01:07 | The agent ended the conversation without directing the customer to pull forward or the next window. |
eRfBYqGFWdiZG43oyP6kVc | UAT #8 | Simulation Run 2026-08-17 01:07 | The agent did not complete the order or direct the customer to pull forward; order capture was incomplete. |
npVZSvG43vS2FHMpCTFggF | UAT #29 | Simulation Run 2026-08-17 01:07 | The agent did not confirm the full order or direct the customer to pull forward; no order completion is evident. |
Failed conversations only; the persona page has all 137.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
787rPLntnU4XCHHaFYiHKM | UAT #69 | RBI Bakeoff | Agent asked an employee to help with order verification, escalating to a human team member. |
7eWv2tggoLKZdtGYbhx5Lt | UAT #42 | RBI Bakeoff | The transcript does not show the agent directing the customer to pull forward or next window after taking the full order… |
WrsSZxTpDhbuYLf5giYFmw | Combo offer after signature chicken | RBI Bakeoff | The agent took the full order but never directed the customer to pull forward or to the next window. |
eyRwxp3L3woS3gHVLQayAM | UAT #2 | RBI Bakeoff | The agent escalated to a human employee and did not direct the customer to pull forward or the next window. |
4hCuhCgrZHoJieawUtC2Qj | UAT #61 | RBI Bakeoff 2026-07-23 | The agent repeatedly indicated an employee would assist, escalating instead of directing the customer forward. |
Cpins8avsExNfCLZG5RBiN | Combo offer after sandwich | RBI Bakeoff 2026-07-23 | Agent handed off the order to an employee instead of directing customer to pull forward. |
fvsymfMsFDnvF6YkjrGrHA | UAT #2 | RBI Bakeoff 2026-07-23 | Agent did not read back the full order and escalated to an employee for assistance. |
hkkARPaCsqZFmiVu9NsidV | UAT #8 | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member multiple times for order confirmation assistance. |
nzDxtTsrSQNNG5SsXWk3No | Combo offer after tenders | RBI Bakeoff 2026-07-23 | The agent escalated the order to a human employee instead of directing the customer to the next window. |
PCfazsyEygesfshuRXDfSe | UAT #61 | RBI Bakeoff 2026-07-24 | Agent escalated the order by saying an employee will assist, not directly sending customer forward alone. |
DNbjiSaZxMdiQXoiPhM4Xv | UAT #10 | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human team member, stating an employee would assist customer. |
NjNgYNbuGqKVUsqvV5khFc | Combo offer after tenders | RBI Bakeoff 2026-07-24 Pass 5 | Agent acknowledged that an employee will assist the customer, indicating escalation to a human. |
bg8hi6NJD6Wod2hr9DZ3C8 | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The conversation shows order capture but lacks agent directing customer to pull forward or next window. |
DSPqeXmv6KWEigMqQ7t5Bq | Combo offer after tenders | Simulation Run 2026-08-17 01:07 | Agent did not capture the order or direct the customer to pull forward; no order completion evident. |
d7s7mK8kQx48dSFoH3RBLZ | UAT #11 | Simulation Run 2026-08-17 01:07 | The transcript does not show the agent directing the customer to pull forward or proceed to the next window after order … |
oVHbknv5ZaVoqfwmcJfdaJ | UAT #10 | Simulation Run 2026-08-17 01:07 | Agent did not capture the full order or direct the customer to pull forward; no completion shown in transcript. |
Failed conversations only; the persona page has all 136.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
8SBCcweYQBnXYEfPTkzx5V | UAT #29 | RBI Bakeoff | Agent escalated to a human team member to assist after user asked for order to be read back. |
AGMMsmZ5zeKXwjXjo6k65A | UAT #9 | RBI Bakeoff | The agent escalated to a human team member to assist with reading back the order, failing to fully complete the order th… |
hdweVSYcbucrB4C89ZTDAM | UAT #9 | RBI Bakeoff | Agent escalated to a human team member to assist with order verification instead of directing customer forward. |
DSvQwWuHREdrCAs3iyktK8 | UAT #11 | RBI Bakeoff 2026-07-23 | The agent said an employee will assist, indicating escalation and no direct direction to pull forward by the agent. |
NCe8QHwSm8jvUpessXVTqm | UAT #9 | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee multiple times without sending customer forward. |
5rAFv4iU94tVmYnBZC5GJt | UAT #2 | RBI Bakeoff 2026-07-24 | Agent escalated the conversation to a human team member instead of directing customer forward. |
5rDHgtRtAjRwerx2JxdpCB | UAT #29 | RBI Bakeoff 2026-07-24 | Agent confirmed order but only entered quantity 1 instead of 12-piece family meal and did not handle full quantity as pe… |
57ymjVhh5DnzSDSZBwUFjK | UAT #2 | RBI Bakeoff 2026-07-24 Pass 5 | Agent did not capture the full order; only the classic wrap combo is in the order, missing blackened tender wrap and man… |
LSkeGYER9oPn9n3hsbYym4 | UAT #10 | RBI Bakeoff 2026-07-24 Pass 5 | Agent did not direct the customer to pull forward; instead, escalated to an employee for assistance. |
h7xGURDgHdzhQMHiNqiyiy | UAT #29 | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human employee instead of completing the order alone and directing customer to pull forward. |
R9xyGryrKUJjTb2DSX2DH6 | Combo offer after signature chicken | Simulation Run 2026-07-20 23:03 (scratch) | The transcript does not show the agent directing the customer to pull forward or the next window. |
F9uCPCAN5hhjDjzVzjnfiW | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | Agent did not complete order or direct to next window; escalated to human employee. |
GYYz2LWKEV9iFimg3m9Reu | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The agent never directed the customer to pull forward or the next window after taking the order. |
TjhAC7FDeshi8EXJs8f6ge | UAT #8 | Simulation Run 2026-08-17 01:07 | Agent escalated to a human by saying 'Let me get an employee to help you out.' |
Failed conversations only; the persona page has all 143.
| Simulation | Test case | Batch | Judge note |
|---|---|---|---|
8uugsoSvaqMaAxo3ko98Ya | UAT #10 | RBI Bakeoff | The agent did not finalize the order and handed off to an employee for assistance. |
KquhRWKPmxH87YTA6SWJih | Combo offer after signature chicken | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
DBGwtbfXs88tk4sYyaz2sS | Combo offer after sandwich | RBI Bakeoff 2026-07-23 | Agent asked 'Is your second side dish?' indicating the order was not fully captured before directing to the window. |
NcG5WLJfyH6WNxiFgt326w | UAT #29 | RBI Bakeoff 2026-07-23 | The agent repeatedly stated that an employee will assist the customer, indicating escalation to a human team member befo… |
3tKQRrLm8ybBTf9FXugAPC | UAT #1 | RBI Bakeoff 2026-07-24 | The agent ordered a Spicy Chicken Sandwich instead of the Ghost Pepper Sandwich the customer requested, so the order was… |
PLV8acSdbN53HmfU6NmDTb | UAT #61 | RBI Bakeoff 2026-07-24 | The agent did not complete the order by themselves but handed off to an employee at the window. |
nmtVzWDH7e4wJ3fA28HGrk | UAT #9 | RBI Bakeoff 2026-07-24 Pass 5 | Agent asked for a second side dish but did not update the order; order not fully captured. |
7q9nqzAFpSwUUuNfn67y2N | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The transcript shows the order but lacks confirmation and direction to pull forward, no indication of order completion o… |
NRnAQqyW8ZgYZV8wepuA3v | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | Agent never confirmed full order completion or directed customer to pull forward; no indication order was finalized with… |
jfkhr4AR5yKXGG7gWNxLQM | Combo offer after signature chicken | Simulation Run 2026-08-16 21:10 | The agent did not explicitly direct the customer to pull forward or the next window after taking the order. |
862ckZW28jBRbtrwmZkpmx | UAT #36 | Simulation Run 2026-08-17 01:07 | The transcript shows the order was partially captured, but the agent did not explicitly direct the customer to pull forw… |
DdxdbJEGX6S2yT6gZiDuc8 | UAT #10 | Simulation Run 2026-08-17 01:07 | The transcript shows no agent response capturing the full order or directing the customer forward. |
MZbJzaBeAaqD7aaZ98HKRq | UAT #11 | Simulation Run 2026-08-17 01:07 | Agent did not direct customer to pull forward or the next window after taking the full order. |
U5TYkPaFmc2sXS6YBixUaS | Combo offer after signature chicken | Simulation Run 2026-08-17 01:07 | The transcript does not show the agent directing the customer to pull forward or the next window after order capture. |
f3hidmZyzHJV5tfmQqFZns | Mid-order quantity change | Simulation Run 2026-08-17 01:07 | The conversation does not include the agent directing the customer to pull forward or the next window after taking the f… |
Failed conversations only; the persona page has all 170.
| Persona | Completion | Passed |
|---|---|---|
| High Background Noise Speaker | 37.8% | 54/143 |
| Confused Customer | 46.2% | 78/169 |
| Impatient Customer | 59.1% | 107/181 |
| Interruptive Speaker | 70.4% | 81/115 |
| Behavioral - Complex-Order | 71.2% | 126/177 |
| Low Volume Speaker | 72.7% | 93/128 |
| Behavioral - Rush | 72.7% | 104/143 |
| Standard Customer | 75.0% | 123/164 |
| Accent - General American | 78.6% | 92/117 |
| Accent - Hispanic/Latino | 79.5% | 268/337 |
| Accent - AAVE | 79.5% | 105/132 |
| Behavioral - Mind-Changer | 82.4% | 108/131 |
| Super Fast Speaker | 84.6% | 99/117 |
| Accent - Louisiana Creole | 85.6% | 95/111 |
| Accent - Southern US | 87.2% | 95/109 |
| Behavioral - Too-Talkative | 89.3% | 125/140 |
Test case names link to the filtered runs list in Coval. Combo-offer scenarios cluster at the bottom. Click a bar to see that case's failed conversations inline.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
5YwXgKJS93LD3qDdUYQeQG | Confused Customer | RBI Bakeoff | Agent escalated to a human after customer was unsure about order confirmation. |
AMQnUUGFMG7Me3CnZ5aNFY | Accent - AAVE | RBI Bakeoff | Agent failed to read back order and escalated to a human team member for help. |
DL5ywZJmjcy29M5bSjw2em | Low Volume Speaker | RBI Bakeoff | The agent ended up saying 'An employee will assist you,' indicating escalation to a human. |
GM669tmg5Yb3GjGj4BFJoZ | High Background Noise Speaker | RBI Bakeoff | The agent did not direct the customer to pull forward; instead, they said 'An employee will help confirm your order,' in… |
L2ATPo3f2C2o7rQQEbfnWv | Impatient Customer | RBI Bakeoff | The agent said 'Employee will assist you,' indicating an escalation to a human team member. |
NK7jRHpBFGjPjic79EteJH | Standard Customer | RBI Bakeoff | Agent failed to read back the order and deferred to an employee for assistance. |
W9pZwBkwFwytgvgi4VNUws | Confused Customer | RBI Bakeoff | The agent escalated to a human multiple times instead of completing order and sending customer forward. |
WobG2LCx386BoD5DtnEMWb | Super Fast Speaker | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward without escalation. |
X7VKH4QMcoNieUuKxS4Kyw | Interruptive Speaker | RBI Bakeoff | Agent escalated to a human team member by stating an employee will assist the customer further. |
YjJP6rfLhzTCmqkxwh7jui | Accent - Hispanic/Latino | RBI Bakeoff | The agent said 'Employee will assist you,' indicating escalation to a human team member. |
ZBLBgDuuU9cgiX38DcLGsJ | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human by saying an employee will assist the customer. |
cScsqL2sK7pZpLHftzspCU | Behavioral - Complex-Order | RBI Bakeoff | Agent escalated to a human team member upon customer's request to read back the order. |
fR3UFvDYs5cu58qKVXo5Dz | High Background Noise Speaker | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward after capturing order. |
hCFCxjw29pgjoZqzeTP3Tc | Accent - AAVE | RBI Bakeoff | Agent changed the order from 5 classic tenders to a 3 Piece Chicken Tenders Dinner without clarifying; order incomplete. |
horBgh7HjezSQB7hCCirey | Impatient Customer | RBI Bakeoff | Agent handed off to an employee instead of directing the customer forward after capturing the full order. |
6zFxG2KaR9mAxYsCQ5XV7H | Interruptive Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee instead of directing customer forward without handoff. |
FwQcA6knF5XDZEFZM4LYqZ | Standard Customer | RBI Bakeoff 2026-07-23 | Agent failed to fully read back the order and deferred to employee assistance, indicating incomplete order capture. |
QqQcW5hMJataJfZfyVFyor | Super Fast Speaker | RBI Bakeoff 2026-07-23 | The agent said 'An employee will assist you,' indicating escalation to a human. |
TzPAFksD8Yf4BHjpWzkhCd | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | The agent escalated to an employee to verify the order instead of completing it themselves. |
Yuxnqthq92An6aRD7qDZi9 | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent handed off to a human team member instead of directing the customer forward after taking the order. |
Showing 20 of 51. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
2UuUTSrsH3NpXjq2WyYBPN | Confused Customer | RBI Bakeoff | Agent could not verify order alone and escalated to human for order verification. |
3HKNLxBr55dwtpaT4rU67M | Accent - AAVE | RBI Bakeoff | Agent said 'I can get an employee to help you verify it,' indicating escalation to a human team member. |
7nU7FMtTRuZeUfgJo4HpBX | Low Volume Speaker | RBI Bakeoff | Agent confirmed order and told customer to pull forward but then handed off to a human employee at the window. |
AGMMsmZ5zeKXwjXjo6k65A | Accent - Southern US | RBI Bakeoff | The agent escalated to a human team member to assist with reading back the order, failing to fully complete the order th… |
Db4jZpvxRZKvZu9t3wnHtL | Impatient Customer | RBI Bakeoff | The agent escalated the order to a human team member by saying 'An employee will assist you.' |
EA8kCDWmqYTZSEkHCB9CkU | Accent - Hispanic/Latino | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
FVxknVXBN86xwBFyNHi6a3 | Accent - General American | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward after capturing order. |
FXFj9jg6YsovEeRmiNVFb4 | Standard Customer | RBI Bakeoff | Agent escalated to a human after taking order and didn't direct customer forward themselves. |
FqH3qVKCzFJPCH4BfZ3mJa | High Background Noise Speaker | RBI Bakeoff | Agent did not direct the customer to pull forward or the next window after capturing the order. |
QgBHGChkg35xpCVpf4uBQw | Interruptive Speaker | RBI Bakeoff | Agent escalated to a human team member instead of directing the customer to pull forward. |
arHoPtSc6LjtGJFM7pN5FP | High Background Noise Speaker | RBI Bakeoff | The agent handed off the order verification to an employee and did not complete the order themselves. |
ccTAHm7KPQxA9TkErBzCQF | Standard Customer | RBI Bakeoff | The agent escalated to a human team member multiple times instead of sending customer forward without handoff. |
eHMYmQK9k6B7M3wW4nsTL2 | Behavioral - Mind-Changer | RBI Bakeoff | Order was not fully captured correctly; combo size was inconsistent and agent did not clearly confirm all details before… |
hdweVSYcbucrB4C89ZTDAM | Accent - Southern US | RBI Bakeoff | Agent escalated to a human team member to assist with order verification instead of directing customer forward. |
hrqcVzKKzAuLCD7cQbHvsu | Confused Customer | RBI Bakeoff | Agent confirmed full order but escalated by saying an employee will assist, indicating a handoff. |
kc6zGK3sugeqJWgcy56ijw | Accent - Hispanic/Latino | RBI Bakeoff | The agent escalated to a human employee for order confirmation multiple times instead of directing the customer to the n… |
3Gk8orKbzA5BV2n5zCviSW | Confused Customer | RBI Bakeoff 2026-07-23 | Agent said 'They'll help with the rest there,' implying escalation to a human team member at the window. |
HDmSAjKSmR6wZXetPgicT3 | Accent - General American | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member to read back the order, not directing customer to next window. |
JTPGTEfRKLCEDMT9fjxU3C | Behavioral - Rush | RBI Bakeoff 2026-07-23 | The agent escalated the order verification to a human employee and did not send customer forward without escalation. |
NCe8QHwSm8jvUpessXVTqm | Accent - Southern US | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee multiple times without sending customer forward. |
Showing 20 of 54. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
25DxMcuZszZLs94D4izC8c | Behavioral - Rush | RBI Bakeoff | Agent did not read back the full order and escalated by saying an employee will assist. |
3Q7JaCgUXJnvS38LiLtRY9 | Impatient Customer | RBI Bakeoff | The order was not fully captured due to a discrepancy in the number of Sprites, and the agent repeated the order incorre… |
8SBCcweYQBnXYEfPTkzx5V | Accent - Southern US | RBI Bakeoff | Agent escalated to a human team member to assist after user asked for order to be read back. |
9uMoNS5CfY3CjB8WtgdRt4 | Confused Customer | RBI Bakeoff | The agent handed off the conversation to a human employee instead of directing the customer to pull forward or the next … |
ADKAMQtnWSmb64cF9RtcCU | Impatient Customer | RBI Bakeoff | Agent escalated to a human team member instead of sending customer forward after order capture. |
Aa75ZNaTSzeQPT5XvCUZJp | Low Volume Speaker | RBI Bakeoff | Agent finalized order with quantity 2 for the meal, but customer only confirmed quantity 1, indicating order not fully c… |
CiLfPzLVdfbX3CtQtXyT6D | Behavioral - Rush | RBI Bakeoff | Agent escalated to a human team member instead of directing customer to pull forward. |
NWCrzEHgKjCGMAdjyjPZwN | High Background Noise Speaker | RBI Bakeoff | Agent escalated the conversation to a human employee instead of directing customer forward after full order. |
QmL3kths2eNuEQLtWyrr5s | Accent - Hispanic/Latino | RBI Bakeoff | The agent said 'An employee will assist you,' indicating an escalation to a human team member. |
RkAH27zjroTHB5wQoxtEX4 | Super Fast Speaker | RBI Bakeoff | Agent failed to capture the correct quantity of Sprites; only one was recorded instead of two, thus order not fully capt… |
RzEBqkUfeDztAQvbw4BFau | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human team member instead of completing the order and sending the customer forward. |
T6rgPekfobFAmsQBWCBxTd | Confused Customer | RBI Bakeoff | Agent escalated to a human employee instead of sending the customer forward alone. |
Y6FLQNZdHKuEZ775jEYMC3 | Interruptive Speaker | RBI Bakeoff | The order was not fully captured; the quantity of the 12-piece meal was only recorded as 1 instead of 2 as ordered. |
ajVXLjSYvZu9Fz4k2uxso4 | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to next window after order. |
ijpFLfBKPTvfZUoKPwi8cd | Low Volume Speaker | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
jW5sj4gJVN9RecmoVkFPCz | Standard Customer | RBI Bakeoff | The order was not fully captured correctly (Sprite quantity was inconsistent), so full order was not confirmed before as… |
nh4sNhAXFfqoBdFVbsuvCs | Interruptive Speaker | RBI Bakeoff | Agent did not fully capture the two Sprites (only one shown) and did not explicitly direct customer to next window witho… |
NcG5WLJfyH6WNxiFgt326w | Behavioral - Too-Talkative | RBI Bakeoff 2026-07-23 | The agent repeatedly stated that an employee will assist the customer, indicating escalation to a human team member befo… |
SrNMwrjQXwPYYWBsK9L9Mi | Super Fast Speaker | RBI Bakeoff 2026-07-23 | The order includes two Sprites but system shows only one Sprite ordered, indicating incomplete capture. |
V5XnWTFfqeCtrm4sjiL9Jj | Behavioral - Rush | RBI Bakeoff 2026-07-23 | Agent repeatedly told customer an employee will assist and never confirmed complete order or directed to pull forward. |
Showing 20 of 46. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
39q2xhhVwTd37yWw5RbMYb | Behavioral - Complex-Order | RBI Bakeoff | Agent escalated to an employee for order verification, not completing order alone. |
5E94Br2Xyvxxd79NaMEa6F | Behavioral - Complex-Order | RBI Bakeoff | Agent failed to read back the order and told the customer to get an employee if it looked wrong. |
BSX6PyD4bzRqBnteziKYd2 | Confused Customer | RBI Bakeoff | Agent escalated to a human by saying 'An employee will assist you' after order confirmation. |
E2j3q5FEM9zoXVYHRhdQme | Low Volume Speaker | RBI Bakeoff | Agent never read back full order and suggested an employee can help, showing incomplete handling. |
LF3uBCaLVSprakLXAEaZbS | Accent - Hispanic/Latino | RBI Bakeoff | The agent escalated to a human by saying 'I'll get an employee to help you.' |
SEcqstDg8dGQTe2uYBfUFV | Super Fast Speaker | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to pull forward/next window after capturing order. |
SsLiSbEL8yUhekVkjwMFd9 | Standard Customer | RBI Bakeoff | The agent escalated the order confirmation to a human employee multiple times without finalizing the order or directing … |
WXafFYP4LYmodicsJHgCTU | Behavioral - Rush | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you.' |
eyRwxp3L3woS3gHVLQayAM | Accent - Louisiana Creole | RBI Bakeoff | The agent escalated to a human employee and did not direct the customer to pull forward or the next window. |
fmyZAbCisqoiEm9HpfuKRD | Impatient Customer | RBI Bakeoff | Agent escalated the conversation to an employee, not sending customer forward directly after full order. |
keSy3A89wUNBFZnCb6DvHC | High Background Noise Speaker | RBI Bakeoff | Agent failed to read back order and escalated to a human team member for assistance. |
mATKcFXWFJSQ2joM7szPeR | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human team member and did not direct the customer to pull forward or the next window. |
BN39pSHnBPgwCaeiWwTxsG | Accent - AAVE | RBI Bakeoff 2026-07-23 | The agent failed to capture the full order correctly; the final confirmed order only includes the frozen lemonade, not t… |
CBEEirBHpsjP2eU2U3o93h | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | Agent did not read back the full order and escalated to an employee for help at the window. |
FeJGVYpvtJSPDkvusGDKcL | Confused Customer | RBI Bakeoff 2026-07-23 | Agent did not read back the full order and stated they would get an employee if there was an issue, indicating escalatio… |
KuurwUdQaQ63XpMEpksHnP | Behavioral - Complex-Order | RBI Bakeoff 2026-07-23 | The agent did not complete the order but repeatedly escalated to a human employee to assist. |
fvsymfMsFDnvF6YkjrGrHA | Accent - Louisiana Creole | RBI Bakeoff 2026-07-23 | Agent did not read back the full order and escalated to an employee for assistance. |
imhymrrgfp2yXEocswZpFq | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent handed off to a human employee for order verification and did not direct customer to the next window. |
5rAFv4iU94tVmYnBZC5GJt | Accent - Southern US | RBI Bakeoff 2026-07-24 | Agent escalated the conversation to a human team member instead of directing customer forward. |
G2ELhLkSGGBuAtTgyBMKeY | Impatient Customer | RBI Bakeoff 2026-07-24 | Agent escalated the customer to an employee without directing them to pull forward or the next window. |
Showing 20 of 46. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
6gv7FeZMXry3Qd2XyoGtzn | Impatient Customer | RBI Bakeoff | Agent failed to read back the full order and only instructed the customer to pull forward. |
7UnjnrisqzeUZeij2Gowka | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human by stating an employee will assist the customer instead of directing them to the next win… |
8uugsoSvaqMaAxo3ko98Ya | Behavioral - Too-Talkative | RBI Bakeoff | The agent did not finalize the order and handed off to an employee for assistance. |
CKEUUeoLkzmtkCTUvmzHrv | Confused Customer | RBI Bakeoff | Agent escalated to an employee instead of directing to pull forward without handoff. |
Nw99udeXtK5Ei4qXqXUkuX | Behavioral - Mind-Changer | RBI Bakeoff | Agent escalated to a human employee to confirm the sandwich availability, failing to complete order independently. |
QgYXeANSjaxZdN4oEurE8T | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human team member by saying 'An employee will assist you.' |
RMz8UL6KWxRKm5wFa7YqBv | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human team member by stating an employee will assist the customer repeatedly. |
VpEhXmZd2WD9KMig6jCata | High Background Noise Speaker | RBI Bakeoff | Agent escalated to a human by saying an employee will assist the customer. |
WgD6fsG3rtiC2Dz4FUDhk7 | Impatient Customer | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you shortly' instead of directing the customer to pull… |
ZJtDSjGcMKfm2AnecCopvb | Behavioral - Complex-Order | RBI Bakeoff | The agent escalated the order to a human employee for assistance instead of completing it themselves. |
ey3T5vRhd4T88qubuAmvxG | Behavioral - Rush | RBI Bakeoff | Agent escalated to a human team member instead of sending the customer forward after order capture. |
jEcAj9NrhDs3riLSHTXrEb | Confused Customer | RBI Bakeoff | Agent deferred order verification to an employee and did not direct customer to pull forward or the next window. |
oPgA5FxqrBgjfjEUJih5B2 | Behavioral - Complex-Order | RBI Bakeoff | Agent handed off to a human team member instead of directing customer to the next window. |
NLjXC6xUwcKMwrjkq8rvZg | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member instead of directing the customer to pull forward/next window. |
P8bSLwf2ySSbvmrpPUgfNe | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | The agent escalated the order to a human employee instead of directing the customer to pull forward. The order was hande… |
SgK4m572F7oKUKcjcQgXUL | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-23 | The agent repeatedly said 'An employee will assist you' instead of directing the customer to pull forward or the next wi… |
TjjxcmYtRRX2LCQfWTXY7y | Confused Customer | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member instead of directing the customer to pull forward/next window. |
26aQqRjiNVAa9fLmcBSQv6 | Behavioral - Rush | RBI Bakeoff 2026-07-24 | Agent escalated to a human to help confirm the order, failing to complete it fully. |
3y9pLtRVwbCnZM6j7DsRvr | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | Agent escalated to a human employee instead of completing order and directing customer. |
6SidWtNFX9dppnNt9HqXEU | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 | The agent could not read back the order and said an employee will assist the customer, indicating escalation to a human. |
Showing 20 of 43. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
6JnfFnqBvcaH6B8677qnJb | High Background Noise Speaker | RBI Bakeoff | The agent escalated the conversation to an employee multiple times instead of directing the customer forward. |
Dp7qYF8s7cHgcUJiaazeWy | Behavioral - Complex-Order | RBI Bakeoff | Agent confirmed order but did not direct customer to pull forward or next window. |
HWxptG9gbMFgeFM9Gno6tj | Accent - AAVE | RBI Bakeoff | Agent escalated to human by saying 'An employee will assist you' instead of sending customer forward alone. |
LTRWCBq3wv2EgeQTFXGp3c | Confused Customer | RBI Bakeoff | Agent escalated to a human multiple times, failing to direct customer to pull forward without handoff. |
MKqfVcdheNqT3RJyw8Hd5e | Confused Customer | RBI Bakeoff | Agent did not direct customer to pull forward or the next window and repeatedly stated an employee would assist. |
NY9Z5bszKzehraVhNd2Gct | Interruptive Speaker | RBI Bakeoff | The agent escalated to a human employee instead of directing the customer to the next window. |
QKCb574txFhqXFdoQiMHzm | Accent - AAVE | RBI Bakeoff | Agent captured the full order but repeatedly stated 'An employee will be right with you' instead of directing to next wi… |
QVtSpUf2YKVsajqZPtkVBH | Interruptive Speaker | RBI Bakeoff | The agent did not complete the order and escalated to a human team member for assistance. |
RDpw7aeufNascPGPMWRUY3 | Impatient Customer | RBI Bakeoff | The agent did not direct the customer to pull forward or the next window after taking the order; no instruction to proce… |
Ve9dW96wYTHsLEeHoW7h7N | High Background Noise Speaker | RBI Bakeoff | The agent escalated the conversation to a human by stating 'An employee will assist you.' |
XHhkDNFudZsL2nuzwVVUa8 | Accent - General American | RBI Bakeoff | The agent incorrectly changed the order to a dinner item, not capturing the customer's full original order. |
YGgSbDVwNtRxXsLaFMEa95 | Standard Customer | RBI Bakeoff | The agent escalated to a human employee and did not complete the order themselves. |
c9kAN9NR2xRBAq7J9ayPam | Low Volume Speaker | RBI Bakeoff | Agent escalated the order by saying an employee will assist the customer. |
jVcSu6wmTBvvAUiJaiakBM | Impatient Customer | RBI Bakeoff | The agent handed off the customer to an employee, so did not complete order alone without escalation. |
jXNQbfvNK4cfb8qQ6y5rnc | Standard Customer | RBI Bakeoff | Agent did not direct customer to pull forward; repeatedly stated 'An employee will assist you,' implying escalation. |
4hCuhCgrZHoJieawUtC2Qj | Accent - Louisiana Creole | RBI Bakeoff 2026-07-23 | The agent repeatedly indicated an employee would assist, escalating instead of directing the customer forward. |
JWpHuC8G9f7gYG7y3Ct7vt | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent escalated to an employee instead of directing customer to next window. |
WbEXyeKLfNMebSr3pNuTFZ | Super Fast Speaker | RBI Bakeoff 2026-07-23 | Agent escalated by stating "An employee will assist you," not directing customer to pull forward/next window. |
XnZc9FQ7uskE5tGe7pSABh | Accent - AAVE | RBI Bakeoff 2026-07-23 | Agent did not direct the customer to pull forward; instead, they said an employee would assist, indicating escalation. |
YynXmhVr8UHShiBsBWxYq5 | Impatient Customer | RBI Bakeoff 2026-07-23 | Agent escalated the conversation to a human by saying 'An employee will assist you' multiple times. |
Showing 20 of 43. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
9GGx8tNJEkYCfgmpogVHd6 | Confused Customer | RBI Bakeoff | Agent confirmed that an employee will assist, indicating escalation and not directing customer forward independently. |
9a3rqyokoB6RVFUhEEtYWb | High Background Noise Speaker | RBI Bakeoff | Agent failed to read back the order and escalated to a human employee for assistance. |
ArtfpAqrwz7JHRit3k4bbx | Low Volume Speaker | RBI Bakeoff | Agent did not complete order or direct customer forward; instead, agent escalated to a human employee. |
DoXBp5QsXvE3gt6m9xQrjA | Accent - General American | RBI Bakeoff | The agent escalated to a human employee for assistance, not sending customer forward without escalation. |
GrHVB3hdUUfLCPqcqHzSvu | High Background Noise Speaker | RBI Bakeoff | Agent does not send customer to pull forward or next window; no direction given after order confirmation. |
HFN3zo9apgbMw9mBELF3Ns | Accent - General American | RBI Bakeoff | Agent escalated the order to a human team member for verification instead of directing the customer forward. |
KquhRWKPmxH87YTA6SWJih | Behavioral - Too-Talkative | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
MJWQWbPQyiu5dJBUUrsDon | Behavioral - Complex-Order | RBI Bakeoff | Agent escalated to a human multiple times without sending customer forward or confirming order completion. |
NeyqLS4uEphBsNxVgBEP3f | Impatient Customer | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member. |
VLtEo3QDrZDNYiNFWhBkhx | Confused Customer | RBI Bakeoff | Agent escalated to a human after failing to confirm the full order and read it back to the customer. |
WrsSZxTpDhbuYLf5giYFmw | Accent - Louisiana Creole | RBI Bakeoff | The agent took the full order but never directed the customer to pull forward or to the next window. |
fwBtmD9RM2CPMG7UwzgPLR | Interruptive Speaker | RBI Bakeoff | Agent could not read back the order and directed customer to screen or employee assistance, not fully completing order t… |
izCEpTnUToz63eL8a7VLAR | Impatient Customer | RBI Bakeoff | The agent confirmed the order but repeatedly stated an employee will assist the customer, indicating escalation. |
jgGBpyZih8JDxLyFaWXxpy | Behavioral - Rush | RBI Bakeoff | The agent stated 'An employee will assist you,' indicating escalation and not completing the order interaction fully alo… |
mUzsUGP5LC3fU4BsCEy2jM | Low Volume Speaker | RBI Bakeoff | Agent escalated to a human employee for assistance instead of directing customer to pull forward. |
Asx9YUMgKSYqkTy4hLk3jc | Confused Customer | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee to verify the order, not completing fully independently. |
KEAk2eeF52K4P7aj6wznmr | Behavioral - Complex-Order | RBI Bakeoff 2026-07-23 | Agent repeatedly mentioned 'An employee will assist you' indicating escalation to a human. |
k6kUM8L9g2r497JbTsn7zB | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | Agent did not complete the full order or direct customer forward; escalated to human multiple times. |
DYLBetiV8vEHwAaH2UBCo8 | Confused Customer | RBI Bakeoff 2026-07-24 | The agent did not complete the order independently and escalated to a human for order confirmation. |
RMyQSdnpW724NzF823ytex | Impatient Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human by saying an employee will assist the customer. |
Showing 20 of 64. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
HdbpxvPPghx23pJ7Jywfxc | Accent - Hispanic/Latino | RBI Bakeoff | Agent confirmed order but escalated to human when unable to read order back to customer. |
LoxJ2uUjKXdLmZy4dKDPJZ | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human by saying 'An employee will assist you.' |
Mh4JrSWLRRkWrj2uNeEKz8 | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human and did not direct customer to pull forward after taking order. |
Wv4RCHMWgMa8kgEBzeUoGn | Confused Customer | RBI Bakeoff | The agent handed off the conversation to a human team member and did not direct the customer to the next window. |
f9op3UXPtESqFWfPDtYRzm | Behavioral - Rush | RBI Bakeoff | The agent said 'No problem. An employee will assist you,' indicating a handoff to a human team member. |
mKjpWp6z3e9qNYs6CMuJhX | Confused Customer | RBI Bakeoff | Agent said 'An employee will assist you', indicating escalation to a human team member. |
nstFbStcCebtsttZgBAmeL | Behavioral - Rush | RBI Bakeoff | Agent escalated to a human employee instead of directing customer forward without handoff. |
ADJAea7YnVHnv2oHsuGmm8 | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human by saying an employee will assist you after order confirmation. |
NW4LNmdM6Sc3ePmfu5DZyp | Behavioral - Rush | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member for order verification instead of directing customer forward. |
PcqdumCD7ebHqzQX8RnxCM | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member with 'An employee will assist you.' |
YXsRTxMJKfvhPDrtfbmTan | Behavioral - Complex-Order | RBI Bakeoff 2026-07-23 | The agent failed to read back the order and escalated to a human employee for assistance. |
dK3N54e8zzPiNL3ogV9C86 | Confused Customer | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee instead of handling the order completion and directing the customer forward. |
EtZgDuXefayU2Yx5bhKArC | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member for order verification before completion. |
J4cjT6KiECcC4g62tA6rNt | Confused Customer | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member by telling customer to wait for an employee. |
RcxV5Yx3Cq7G4P7aDzD5i5 | Impatient Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human employee instead of sending the customer forward after taking the order. |
dDezyevYC5oDSBVHxQu6xi | Accent - AAVE | RBI Bakeoff 2026-07-24 | The agent escalated to a human by saying 'An employee will assist you,' not sending the customer forward directly. |
mNz6yqFUndPPzxftMKznEQ | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 | The agent did not complete the order alone and escalated to a human team member to assist further. |
4KZxjEvba4FHRzyuXNfA8a | Behavioral - Rush | RBI Bakeoff 2026-07-24 Pass 5 | Agent was unable to read back the order and suggested human help, indicating escalation. |
6mVEWpDDnpM8KL5yk6NWLJ | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to an employee instead of completing order and directing customer forward. |
9YRkLvHYRBJFngarQgcUUq | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human by saying 'An employee will assist you.' after order confirmation. |
Showing 20 of 35. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
74zFvHx89ADjtc24t8EbJ2 | Confused Customer | RBI Bakeoff | The agent escalated to a human team member due to customer's difficulty with the screen and lack of readback capability. |
CXT2vtwwmbCPGYLoHkC65V | Accent - Hispanic/Latino | RBI Bakeoff | Agent escalated to a human team member to help verify the order before payment. |
DeR3Fs93y6m6PpWnv5H2fE | Behavioral - Complex-Order | RBI Bakeoff | Agent failed to capture customer's requested order details correctly and escalated to a larger combo with modifications. |
ES4AbgQErFhtU4jaUSqjnw | High Background Noise Speaker | RBI Bakeoff | The agent did not complete the order by directing the customer to pull forward; instead, they escalated to a human team … |
J9EZnexVDGiuAXxxVx3dWD | Interruptive Speaker | RBI Bakeoff | The agent escalated to a human team member for order verification instead of sending the customer forward. |
MfqJpvw7iDyK2jhC3FQfkD | Interruptive Speaker | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer forward without escalation. |
Q3c3eLvKiVkiBbeTrcM8zu | Accent - Hispanic/Latino | RBI Bakeoff | Agent failed to read back order and escalated to an employee for assistance, not sending customer forward. |
SKmu5fyiSiEgMugKqCvCht | Behavioral - Mind-Changer | RBI Bakeoff | Agent escalated to a human team member instead of directing customer to pull forward without escalation. |
WuomVFj8gaUGwAxkRbDGPo | Confused Customer | RBI Bakeoff | The agent escalated to a human employee instead of sending the customer to the next window. |
YUUfL4vEUQFLjZEqtwSLaL | Impatient Customer | RBI Bakeoff | Agent failed to complete order and escalated to a human team member for assistance. |
f4YizsNNYXWxfegqC75hUH | Impatient Customer | RBI Bakeoff | Agent escalated the order to a human team member instead of directing customer forward. |
jPRbyGcw8gQt8skjXT6aDv | High Background Noise Speaker | RBI Bakeoff | Agent said 'An employee will assist you,' indicating escalation to a human team member. |
5iZvibFE8hdVKSEAJS75Kh | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent escalated the order to a human team member instead of sending the customer forward themselves. |
GDdscBKN8GGRAgoBL33EAt | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member instead of completing and sending the customer forward. |
UwtUKyMr4Uv7SzvepffGYi | Confused Customer | RBI Bakeoff 2026-07-23 | The agent did not direct the customer to pull forward or the next window but instead escalated to a human employee. |
bXuNgUS75xRxfaf3Svkcpd | Impatient Customer | RBI Bakeoff 2026-07-23 | The agent did not direct the customer to pull forward or the next window but escalated to a human team member. |
hkkARPaCsqZFmiVu9NsidV | Accent - Louisiana Creole | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member multiple times for order confirmation assistance. |
SpwgTGgJtfpLo3bfRcV98L | Confused Customer | RBI Bakeoff 2026-07-24 | Agent stated 'Employee will assist you', indicating escalation to a human team member before directing the customer forw… |
nqY9PGpmHuA8GpnsDCW2mk | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | The agent escalated to a human by stating an employee will assist and did not complete the order by reading it back or d… |
7riuJnj9EsfUiUUXDHXAdH | Accent - AAVE | RBI Bakeoff 2026-07-24 Pass 5 | Agent rescinded initial order choice and repeated sauces without completing the new order before sending customer forwar… |
Showing 20 of 36. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
5qwMkniwwoBXDjFqZjbDH5 | Impatient Customer | RBI Bakeoff | The agent repeatedly stated an employee would assist, thus escalating to a human and not completing order alone. |
AwvKmeip5xUE6Ku8XYL5Ss | Confused Customer | RBI Bakeoff | The agent escalated to a human by stating 'An employee will assist you' after the customer was confused. |
DFpYUqgufGpz9Q9R7tZJcw | Behavioral - Complex-Order | RBI Bakeoff | Agent escalated to a human employee instead of directing customer forward. |
NT2u8KouYNJpqb7d9PmK37 | Accent - General American | RBI Bakeoff | The agent said they can't read back the order and implied escalation to a human if incorrect. |
XfjUww9JTAV7umTVBa9kp8 | Confused Customer | RBI Bakeoff | The agent escalated to a human when they said an employee will help and couldn't read back the order. |
aMLQVa8AV2jhvcsRJNZJDr | Behavioral - Mind-Changer | RBI Bakeoff | The agent initially told the customer to pull forward before the order was complete, and the customer changed the order … |
aNEcZgFv6ECc9SBTEBPpue | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human team member multiple times instead of completing the order and directing the customer for… |
ehMiTJQScTek2aCa8dnWxi | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human team member instead of completing the order and directing the customer forward. |
oAjPgFzupR3Z7xhGEemVYk | Accent - AAVE | RBI Bakeoff | Agent said 'Employee will assist you,' indicating escalation to a human team member. |
7nwTK5uxnM3muZmRZvbdsp | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-23 | Agent escalated to a human by saying 'An employee will assist you' after taking the full order. |
CCyEdngGiZUikhXU2yiwk5 | Impatient Customer | RBI Bakeoff 2026-07-23 | The agent repeatedly failed to complete the order and continuously escalated to a human employee multiple times. |
Cpins8avsExNfCLZG5RBiN | Accent - Louisiana Creole | RBI Bakeoff 2026-07-23 | Agent handed off the order to an employee instead of directing customer to pull forward. |
DBGwtbfXs88tk4sYyaz2sS | Behavioral - Too-Talkative | RBI Bakeoff 2026-07-23 | Agent asked 'Is your second side dish?' indicating the order was not fully captured before directing to the window. |
RuTtQrmmLf27c3v4YWyL3b | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member instead of completing order and directing customer forward. |
V6hq4NdxWXGRJ5fdMQsKz2 | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human by saying 'An employee will assist you.' |
9NMhouX5wgaxSWwMXyctgD | Interruptive Speaker | RBI Bakeoff 2026-07-24 | Agent could not read back the order and deferred to employee assistance, indicating escalation. |
AoaPp5jeYbGw4XSGvyAD8u | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-24 | The agent escalated to a human team member instead of directing the customer to pull forward or the next window. |
JXnEp9qujCbh6geQjWq9V4 | Low Volume Speaker | RBI Bakeoff 2026-07-24 | Agent did not complete the order; instead, said an employee will assist, indicating escalation. |
MkXPL7qRTDMr4MDriCzBfz | Impatient Customer | RBI Bakeoff 2026-07-24 | Agent did not send customer forward but repeatedly stated 'An employee will assist you,' indicating escalation. |
hgZKp9n4D5DL6nEwcHyFVk | Confused Customer | RBI Bakeoff 2026-07-24 | The agent said 'Needed, an employee will assist you,' indicating escalation to a human. |
Showing 20 of 34. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
787rPLntnU4XCHHaFYiHKM | Accent - Louisiana Creole | RBI Bakeoff | Agent asked an employee to help with order verification, escalating to a human team member. |
DFWFmvNDYNCcv9NAEE6SqX | Behavioral - Mind-Changer | RBI Bakeoff | Agent escalated to a human team member instead of directing customer forward after taking order. |
EsFDbbGBBZYkEk7n3sgfuq | Standard Customer | RBI Bakeoff | Agent escalated to a human employee instead of completing the order and directing the customer forward. |
MogikpDVzrWP2SchiBQbue | Behavioral - Rush | RBI Bakeoff | The agent escalated to a human after the customer asked for a read back of the order and could not visually confirm it. |
ToRACtz5RohZUNmaJTP4gn | High Background Noise Speaker | RBI Bakeoff | Agent states 'I can help at the window. Employee will assist you,' indicating a handoff to a human team member. |
Tw28c76BuybrZG9yjPQSwR | Behavioral - Complex-Order | RBI Bakeoff | Agent failed to read back full order and escalated to a human before sending customer forward. |
ULHkYRxYbQMDoxewqMs7n3 | Low Volume Speaker | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer to pull forward. |
UVwUpUu2GC6QXwEiJEXLPS | High Background Noise Speaker | RBI Bakeoff | Agent completed order but escalated by saying 'An employee will assist you,' not directing customer to pull forward or n… |
eR8WysDFpUfdiPbB3Vf4Z9 | Confused Customer | RBI Bakeoff | Agent did not direct customer to pull forward or the next window; instead, the order was escalated to a human employee. |
fVbE7NSAVuRz7wpbnHZKg4 | Confused Customer | RBI Bakeoff | Agent took the full order but escalated to an employee instead of sending customer to the next window. |
gXqxN9aKm8cFfCSZvk7wiU | Low Volume Speaker | RBI Bakeoff | The agent did not direct the customer to pull forward/next window and instead escalated to a human employee. |
iRu8gSv5t9GcYnJ7e4EzwN | Impatient Customer | RBI Bakeoff | The agent did not direct the customer to pull forward or the next window; instead, the agent told the customer to check … |
38rcejCoGuJhnCxogWSQyZ | Impatient Customer | RBI Bakeoff 2026-07-23 | Agent escalated the conversation to a human employee and did not complete the order themselves. |
75YQVG2BwKwBFdRs2BwGd5 | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-23 | The agent failed to complete the order and did not direct the customer to pull forward or the next window. |
fUiciuyisjtRrP7SWSHem5 | Confused Customer | RBI Bakeoff 2026-07-23 | The agent never captured a full order nor directed the customer to pull forward or the next window. |
ibkXkkQFhqfiY7sE5Pp7Tt | Low Volume Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human to verify the order, not directing customer forward themselves. |
mmMGRYoQ7AaBLW55do3Ja9 | Behavioral - Complex-Order | RBI Bakeoff 2026-07-23 | Agent failed to read back the order and directed customer to pull forward for employee assistance. |
nwGvD3EduoTJb7ZjnvKYrC | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human and did not complete the order fully by themselves. |
GLviFybjdbEqXkTg2p2LyC | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | The agent escalated to a human employee after the customer requested order confirmation. |
PpRXgRg8xUNXRuyDNPdQNp | Confused Customer | RBI Bakeoff 2026-07-24 | Agent did not send customer forward but escalated to a human for order verification. |
Showing 20 of 32. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
Av6BQ8Qm2wEyCD5cGH3yDp | Impatient Customer | RBI Bakeoff | The agent escalated the conversation by saying 'An employee will assist you.' |
Kvrczp74hr5BwtEUHT68N8 | High Background Noise Speaker | RBI Bakeoff | Agent escalated to a human employee to verify the order and assist the customer. |
MAYfprdnawRdU4QxacycXn | Impatient Customer | RBI Bakeoff | The agent repeatedly said an employee would assist, indicating escalation to a human team member. |
PzTsXLDVHzca7EpHfXkzrX | Confused Customer | RBI Bakeoff | Agent escalated to a human employee instead of directing customer to next window after order was captured. |
VvaQ2397RQqBf7xZZD4pnC | Accent - General American | RBI Bakeoff | Agent instructed customer to pull forward but mentioned an employee will assist, implying handoff to human. |
WokERmgWVuSkgfXEKb7qxZ | High Background Noise Speaker | RBI Bakeoff | Agent escalated by stating 'An employee will assist you' instead of completing the order and directing customer to next … |
8YVTFJAS5xUxQBRSBpCTZj | Confused Customer | RBI Bakeoff 2026-07-23 | The agent escalated to a human employee to verify the order and assist the customer, not directing them to the next wind… |
NrUhuZ4jqkBsHCxiSwvGA5 | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member for order verification instead of directing the customer forward. |
TvZr3txHnaQYuJR3NUxBrH | Interruptive Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to another employee to assist with order confirmation, not completing entire order themselves. |
ZbJT8HG6tAG5JSpKGMEhz2 | Behavioral - Complex-Order | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member stating 'An employee will assist you'. |
cUxzVrmoXHDaKHzpe78iPr | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human by saying 'An employee will assist you' after order was taken. |
dmAFv6oQ8GWM769WUYxndv | Impatient Customer | RBI Bakeoff 2026-07-23 | Agent escalated to human by saying 'An employee will be right with you.' |
QFMkoURoxz9exh4rPo7TcH | Confused Customer | RBI Bakeoff 2026-07-24 | The agent did not direct the customer to pull forward or the next window, but said 'An employee will assist you,' implyi… |
cfni4H85i9EDdNmfiEtSvn | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 | Agent escalated to a human team member to assist with order confirmation before sending customer forward. |
hAVa6KL2wVSRrcWaZ4vLQN | Interruptive Speaker | RBI Bakeoff 2026-07-24 | Agent said 'An employee will assist you,' indicating handoff to a human team member before directing customer forward. |
4FR4jwHRfqUvqgAJrF2XPT | Impatient Customer | RBI Bakeoff 2026-07-24 Pass 5 | Agent failed to confirm order details verbally and ended conversation without directing customer forward. |
5fcyfbcdGdE9F9j46pEUCX | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | Agent confirmed order but said 'An employee will assist you,' indicating escalation to a human. |
75NkfSQrS7ozRZzwQk4myT | Confused Customer | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human employee rather than completing order and directing customer forward. |
Cw39cQ8Hw8FM6qGuwn9cAu | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human team member instead of directing the customer to pull forward after taking the full order… |
kzpvRZd7P7BBWmC3iXSm3o | Accent - AAVE | RBI Bakeoff 2026-07-24 Pass 5 | The agent did not direct the customer to pull forward or the next window after taking the full order. |
Showing 20 of 39. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
9hKZYye4yrEdJdMXP4mLGd | Accent - Hispanic/Latino | RBI Bakeoff | The agent escalated to a human team member instead of directing the customer to the next window. |
9p9yuQFfk27X3oGZiVPeWu | Interruptive Speaker | RBI Bakeoff | The agent failed to read the order back and suggested a human could assist, indicating a handoff or escalation potential… |
H6CEFryaPHpnMgBci2cCUS | High Background Noise Speaker | RBI Bakeoff | Agent repeatedly stated an employee would assist, indicating escalation to a human team member. |
KTnXSSmhHgm2FBn5RWda9b | High Background Noise Speaker | RBI Bakeoff | The agent escalated the conversation to a human team member instead of directing the customer to pull forward. |
Qhboofo7T9Fx9C64Ef5sYG | Behavioral - Rush | RBI Bakeoff | Agent escalated to a human team member by saying an employee will assist the customer. |
W7XH6rpEKrfUZuzEJNbEnX | Impatient Customer | RBI Bakeoff | The agent escalated to a human team member multiple times instead of completing the order and directing the customer to … |
YqokBfEmpFhzgBTLzTzS2h | Confused Customer | RBI Bakeoff | Agent escalated to a human after confirming the order and did not direct customer to next window. |
fos7hhjNTVRxLq5oyfVRwN | Impatient Customer | RBI Bakeoff | Agent escalated to human after refusing to read back order despite full order capture. |
WacTafaXqzoAJKNzseaDNZ | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent escalated to a human employee rather than directing the customer to pull forward or the next window after order co… |
eUvrpzDVVhtVhrAVFrhkps | Super Fast Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human by saying an employee will assist the customer. |
epoQWeYNDmjC3Gse9oeJYy | Confused Customer | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member instead of directing the customer to pull forward or the next window. |
oRuGKRzL7fpp7SSTKeiGTv | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member instead of directing the customer to pull forward after taking the full order… |
6ayoJPdJdWkUi6kxwk78Gy | Accent - General American | RBI Bakeoff 2026-07-24 | Agent did not read back order and referred customer to an employee, indicating escalation. |
7PqsQy33jkUtLJLdXrUejB | Confused Customer | RBI Bakeoff 2026-07-24 | Agent did not direct customer to pull forward or next window; instead, escalated to employee assistance repeatedly. |
A3NoDbhCqKc6gQAMUkBUfj | Impatient Customer | RBI Bakeoff 2026-07-24 | Agent escalated to a human multiple times instead of directing customer to next window. |
RAyEvRBHYXovj7iyL4Saaq | Behavioral - Rush | RBI Bakeoff 2026-07-24 | Agent repeatedly escalated to a human employee instead of directing customer forward without handoff. |
TKDtcEnLqjtqRAFtX8VtPF | Standard Customer | RBI Bakeoff 2026-07-24 | The agent did not read back the full order and deferred to a human employee for verification multiple times. |
Vcfuk2qGAUPkfzBVr7rzhM | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | The agent said 'An employee will assist you,' indicating escalation to a human team member. |
gquJGMZQFT9JCGvvPSe4Pr | Super Fast Speaker | RBI Bakeoff 2026-07-24 | The agent prematurely told the customer to pull forward before fully confirming the premium strawberry frozen lemonade a… |
4qoqeRvqXxvXaXfaGYu72w | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human employee to verify the order instead of completing it themself. |
Showing 20 of 31. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
6dY6NBjup6EyayNDD5T4Af | Impatient Customer | RBI Bakeoff | Agent said, 'Let me get an employee to help you out,' indicating escalation to a human team member. |
7eWv2tggoLKZdtGYbhx5Lt | Accent - Louisiana Creole | RBI Bakeoff | The transcript does not show the agent directing the customer to pull forward or next window after taking the full order… |
9abs7kYNdqnu4XL9fLSFjY | Confused Customer | RBI Bakeoff | The agent did not complete the order themselves and referred the customer to an employee to verify the order. |
U9QMYnopq8EUmXt7vmzg9f | Accent - AAVE | RBI Bakeoff | Agent could not read back order and deferred to employee assistance, indicating escalation or bail-out. |
ZfKNoYJYMZtMeV5w4n7zPe | High Background Noise Speaker | RBI Bakeoff | The agent escalated to a human by stating an employee will assist the customer. |
cJJwcB2Xo8ZJ9CffxQMeFF | Confused Customer | RBI Bakeoff | Agent escalated to a human team member instead of directing the customer to pull forward. |
duJRLxY6i7YYjYkRNbUPDQ | Standard Customer | RBI Bakeoff | The agent escalated to a human team member instead of completing the order and sending the customer forward. |
eocPooQp5X99aKRv5WKYaB | Impatient Customer | RBI Bakeoff | Agent escalated to a human team member twice and did not complete the order alone. |
k9mqx7ZKBVkgV8qZzArXLx | High Background Noise Speaker | RBI Bakeoff | Agent escalated to a human employee for order verification instead of directing customer forward. |
6eg7w6t6PtehbHVa2rDS5G | Confused Customer | RBI Bakeoff 2026-07-23 | Agent completed order but escalated to a human employee instead of directing to next window. |
HWp5Z9uf3TVrycno7xTDZc | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member instead of directing the customer forward after order capture. |
VRuFNWaLkFFee3EfqqR83R | Super Fast Speaker | RBI Bakeoff 2026-07-23 | The agent escalated to a human employee to verify the order instead of completing it independently. |
4ZYPMs7Y2EKcT3qXDmuo7Q | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-24 | Agent couldn't read back the full order and suggested employee help, indicating incomplete autonomous handling. |
Qbx9ThzRXK7e72qyX4bW9p | Accent - AAVE | RBI Bakeoff 2026-07-24 | Agent did not read back the full order and escalated to a human for assistance. |
VZ3rvexTSqRUk4cx9e2JKr | Confused Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human team member by stating "Employee will assist you." |
ioabvvTQRYLMJ6NDqhFuFk | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | Agent escalated to a human by stating an employee will assist multiple times, not directing to next window. |
8trjuCiRv2XtYqijYbkdmi | Impatient Customer | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human employee instead of completing the order and directing the customer forward. |
E6V2GyUPfksPUD5QyJX7f9 | Confused Customer | RBI Bakeoff 2026-07-24 Pass 5 | Agent stated 'An employee will assist you,' indicating escalation to a human team member. |
hBf6GkghocFPSKzrA4Ezss | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-24 Pass 5 | Agent did not direct customer to pull forward or next window; instead, agent said an employee will assist. |
khiG7HdqYeLMirBzGJjw7G | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 Pass 5 | Agent did not read back the full order and referred to an employee for assistance. |
Showing 20 of 30. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
T6iatXuXt49gvSSBL3hA2k | Confused Customer | RBI Bakeoff | The agent escalated to a human employee instead of directing the customer forward after taking the full order. |
ZkrjYEE4TGcxihZfidJbqg | High Background Noise Speaker | RBI Bakeoff | The agent confirmed the full order but escalated to a human employee instead of directing the customer to the next windo… |
dLrXgUa8M29ZC5uiTaw82f | Accent - AAVE | RBI Bakeoff | The agent escalated to a human team member rather than directing the customer to pull forward. |
ngSqCHrPPi5xSAbutuPhf6 | Low Volume Speaker | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member. |
DSvQwWuHREdrCAs3iyktK8 | Accent - Southern US | RBI Bakeoff 2026-07-23 | The agent said an employee will assist, indicating escalation and no direct direction to pull forward by the agent. |
DzPQcP7FUKjdcZJVWEA2mD | Behavioral - Rush | RBI Bakeoff 2026-07-23 | The agent said an employee will assist, indicating escalation to a human team member. |
Q98gcqqkbqGpDCHTwJYeEv | Accent - General American | RBI Bakeoff 2026-07-23 | The agent did not read back the order and escalated to a human team member for assistance. |
Qq8tZc6eRtDyuztZnTVXSn | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent did not direct the customer to pull forward or the next window and escalated to a human by stating an employee… |
eaTcpw3wqYbZCRM4GZi6ka | Accent - Hispanic/Latino | RBI Bakeoff 2026-07-23 | Agent escalated to a human by saying an employee will assist customer, failing to complete order and direct forward. |
9oKMtxV4HFLpPzhGufMYCK | Impatient Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human team member for assistance multiple times instead of completing the order. |
AfMPfPAxKpZfLmPK2mVHzL | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | Agent escalated to a human employee instead of completing the order and directing customer forward. |
NwycfGcSndNBdDkm3VTrRT | Confused Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human by stating 'An employee will assist you' and directing the customer to wait, not sending … |
L4GyZJj6ZjLeJxsim2cpV9 | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to an employee assisting the customer in person, not directing to next window. |
LR6xRveTLm5YLKcrc3o5Uw | Behavioral - Rush | RBI Bakeoff 2026-07-24 Pass 5 | Agent handed off to an employee instead of directing customer to pull forward. |
Xz2jp6DSpi2EPjDps9j8NB | Behavioral - Mind-Changer | RBI Bakeoff 2026-07-24 Pass 5 | Order was fully captured, but agent instructed to pull forward without confirming fries size as medium (stayed regular). |
aefxgEeVHzzRAjZSfmqSCu | High Background Noise Speaker | Simulation Run 2026-07-20 22:35 (scratch) | The agent handed off the order to an employee instead of directing the customer to pull forward. |
mxEQ2BvhiUmbhuzaRuCtdR | Impatient Customer | Simulation Run 2026-07-20 22:35 (scratch) | Agent escalated to a human employee multiple times and did not complete the order or direct the customer forward. |
2WL4Gc83b4brtEJJPt7Re7 | Accent - AAVE | Simulation Run 2026-07-20 23:03 (scratch) | The agent said 'Employee will help fix that,' indicating escalation to a human. |
gkr7vyP2V7yPeGujgvFoyP | Accent - Hispanic/Latino | Simulation Run 2026-07-20 23:03 (scratch) | Agent escalated to a human by stating 'An employee will assist you.' |
aZdZhwE4jYSNwwYReKhGdA | Behavioral - Rush | Simulation Run 2026-07-20 23:58 (scratch) | The agent did not direct the customer to pull forward or the next window but escalated the order to a human employee. |
Showing 20 of 27. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
Lp9GjC3s76QDdHMvMp6B2j | Accent - Hispanic/Latino | RBI Bakeoff | The agent escalated to a human team member by stating an employee would assist the customer. |
VrcVPMjJrWbupJcaP7aaG5 | Confused Customer | RBI Bakeoff | Agent failed to read back the full order and escalated by stating an employee would assist the customer. |
bfsknDbidbtLyFnMTpqQpQ | Low Volume Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human team member, saying 'An employee will assist you.' |
bhN63QEE2oecgRbg9VRJ4o | Impatient Customer | RBI Bakeoff 2026-07-23 | The agent escalated to a human team member to assist the customer instead of completing the order and directing them for… |
ivSfUBZH62B9JkPj6uqA6i | Confused Customer | RBI Bakeoff 2026-07-23 | The agent escalated to a human when unable to read back the order, failing to complete the order themselves. |
Cv6ntf7S6DBbDMridUjAnQ | Confused Customer | RBI Bakeoff 2026-07-24 | The agent escalated to a human team member by stating an employee will assist the customer. |
JhEyShhUtuYrSVWDfjJ5qy | High Background Noise Speaker | RBI Bakeoff 2026-07-24 | Agent did not complete the order; escalated to a human team member for assistance. |
8uDW5AWTieqisEcbnRj6Q5 | Behavioral - Complex-Order | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human team member after failing to read back the full order accurately. |
H4CEnJMgJGMbWUhqcDY8oy | Confused Customer | RBI Bakeoff 2026-07-24 Pass 5 | Agent did not direct customer to pull forward or next window; agent said an employee will assist. |
VStaSzuWAkfQeLfmxUREEB | Impatient Customer | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human team member instead of directing the customer to pull forward. |
aYjcjDnj2iBZhGVtJKYfuZ | Low Volume Speaker | RBI Bakeoff 2026-07-24 Pass 5 | The agent did not direct the customer to pull forward or next window after taking the order. |
fTNX6Sv3PcikQSoHDN5vXL | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human by saying "An employee will assist you." |
h4nMMVSVBAdcfegRV4pJRc | Behavioral - Rush | RBI Bakeoff 2026-07-24 Pass 5 | Agent escalated to a human team member instead of directing customer to next window. |
aTD2p3mZGJ9R7baxeGKa7r | High Background Noise Speaker | Simulation Run 2026-07-20 22:35 (scratch) | The agent escalated to a human team member multiple times and did not complete the order fully themselves. |
JLexbjZxHm9qgaGnJy994Q | Accent - General American | Simulation Run 2026-07-20 23:03 (scratch) | Agent repeatedly said 'An employee will assist you,' indicating escalation to a human. |
52B7YGvu3r5CoRnRQNn7X2 | Behavioral - Mind-Changer | Simulation Run 2026-07-20 23:58 (scratch) | Agent confirmed order but then said 'An employee will assist you,' indicating escalation to a human. |
jzJ9QLzHgJqZQbimrRQwCG | Behavioral - Complex-Order | Simulation Run 2026-07-20 23:58 (scratch) | The agent did not complete the order and repeatedly escalated to a human employee for assistance. |
n4gmoKojMfqKQ5d4iMFbwK | Standard Customer | Simulation Run 2026-08-16 21:10 | The agent did not confirm the full order or direct the customer to pull forward or the next window. |
CUZPVPjih2fqBusxQpHaho | Impatient Customer | Simulation Run 2026-08-17 00:34 | The agent escalated to a human by saying they would get an employee to help. |
PKmSveh4e2ePueDtnR7WRm | Confused Customer | Simulation Run 2026-08-17 00:35 | The agent escalated to a human employee instead of completing the order and directing the customer to the next window. |
Showing 20 of 23. Failed conversations only.
| Simulation | Persona | Batch | Judge note |
|---|---|---|---|
2VnSDr8hjGzG52bK7GbNus | Accent - Hispanic/Latino | RBI Bakeoff | The agent did not direct the customer to pull forward or the next window, instead said an employee will assist you (esca… |
5Yv8CvVdWRV2s5BvkT5mB4 | Accent - General American | RBI Bakeoff | The agent said 'An employee will assist you,' indicating escalation to a human team member before directing to the windo… |
5czNDH2DsaV6ERMkiBFZbg | Impatient Customer | RBI Bakeoff | Agent escalated the order by saying 'Let me get an employee to help you out.' |
FoZpDqPjUDEEP52h3TdVT7 | Accent - Hispanic/Latino | RBI Bakeoff | The agent did not complete the order or direct the customer to the next window; they escalated by saying an employee wil… |
Lh9ua2hG87bwcbBK4SrmqB | Low Volume Speaker | RBI Bakeoff | Agent did not complete the order and escalated to a human employee for assistance. |
cfXeQFyRTU9QvyXjZuPHPT | Impatient Customer | RBI Bakeoff | Agent stated 'And employee will assist you,' indicating escalation to a human team member. |
jTR228Lcah4bspPLWu5GXQ | High Background Noise Speaker | RBI Bakeoff | The agent repeatedly deferred to an employee for assistance and did not take the full order or direct the customer forwa… |
BCYd2taWD9nPWtxYyGhcPq | Low Volume Speaker | RBI Bakeoff 2026-07-23 | Agent escalated to a human after taking the order, not sending customer forward directly. |
CjvFS7wBFgM3xmgNxJm3V6 | High Background Noise Speaker | RBI Bakeoff 2026-07-23 | The agent handed off the order to an employee instead of directing the customer to pull forward. |
is9D6ybbKXpJrv6fJTkFYj | Behavioral - Rush | RBI Bakeoff 2026-07-23 | Agent escalated to human assistance for Ghost Pepper sandwich issue. |
3tKQRrLm8ybBTf9FXugAPC | Behavioral - Too-Talkative | RBI Bakeoff 2026-07-24 | The agent ordered a Spicy Chicken Sandwich instead of the Ghost Pepper Sandwich the customer requested, so the order was… |
CpkZKkPQeYpTN3GSUKoZQK | Accent - General American | RBI Bakeoff 2026-07-24 | The agent escalated to a human employee multiple times to assist with order verification. |
ThUWxTDBaQaBGVCML7VD8K | Confused Customer | RBI Bakeoff 2026-07-24 | The agent could not read back the order and directed the customer to wait for an employee, indicating escalation. |
J688aB9ZNaYtWMzZpH5qDN | Impatient Customer | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human by repeatedly stating an employee will assist the customer. |
h8zgnCK3dM97RusuBrWhdF | High Background Noise Speaker | RBI Bakeoff 2026-07-24 Pass 5 | The agent escalated to a human team member for order verification instead of directing the customer to the next window. |
4STd2TwmSrDXXvQBLsUt69 | High Background Noise Speaker | Simulation Run 2026-07-20 22:35 (scratch) | The agent escalated by saying 'An employee will assist you' instead of directing to next window. |
Vc4KLMiAWXdaCfT7rBXvWy | Low Volume Speaker | Simulation Run 2026-07-20 22:35 (scratch) | The agent escalated to a human by stating 'An employee will assist you.' |
VgtP3jki2aueVB5Ue7Zzqc | Confused Customer | Simulation Run 2026-07-20 23:58 (scratch) | Agent escalated to a human by saying 'Employee will assist you' and instructing customer to wait for assistance. |
Z8HTEap6pizLXzdEuKpEr2 | Confused Customer | Simulation Run 2026-08-16 23:53 | Agent escalated to a human employee after capturing the order but did not direct the customer forward without handoff. |
7JCtcEbn7FACdc9Agiw42k | Low Volume Speaker | Simulation Run 2026-08-17 01:07 | The agent never directed the customer to pull forward or the next window after taking the order. |
Showing 20 of 25. Failed conversations only.
| Test case | Completion | Passed |
|---|---|---|
| Combo offer after tenders | 61.7% | 82/133 |
| UAT #9 | 62.8% | 91/145 |
| UAT #29 | 64.3% | 83/129 |
| UAT #2 | 64.6% | 84/130 |
| UAT #10 | 68.4% | 93/136 |
| UAT #61 | 68.8% | 95/138 |
| Combo offer after signature chicken | 69.8% | 148/212 |
| Alternative side offering | 72.4% | 92/127 |
| UAT #8 | 73.7% | 101/137 |
| Combo offer after sandwich | 74.4% | 99/133 |
| UAT #69 | 74.6% | 94/126 |
| UAT #64 | 75.8% | 122/161 |
| UAT #36 | 75.8% | 97/128 |
| UAT #42 | 77.1% | 101/131 |
| UAT #11 | 79.9% | 107/134 |
| Mid-order quantity change | 84.0% | 121/144 |
| UAT #1 | 85.1% | 143/168 |
Four official passes, no movement. Whatever changed between them did not move the headline metric. Click a bar to see that batch's failed conversations.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
6nFcij6keL6MVkDJbBP8PE | Accent - Hispanic/Latino | UAT #10 | Agent escalated to a human team member instead of directing the customer to pull forward. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
iK7TyHPeTWJi5rK3F4VhvL | Accent - Hispanic/Latino | UAT #2 | Agent said they would get a manager and employee to assist, indicating escalation to a human team member. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
jqQsY6qmvYedPvrNsEWbSX | Behavioral - Rush | UAT #9 | The agent asked for a second side dish, showing the order was incomplete initially, indicating the order was not fully c… |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
PKmSveh4e2ePueDtnR7WRm | Confused Customer | Mid-order quantity change | The agent escalated to a human employee instead of completing the order and directing the customer to the next window. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
CUZPVPjih2fqBusxQpHaho | Impatient Customer | Mid-order quantity change | The agent escalated to a human by saying they would get an employee to help. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
czxUYoeyZdiCQbKzZ2qywA | Standard Customer | UAT #2 | The transcript ends before the agent directs the customer to pull forward or next window without escalation. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
MbXXmyf5Bmm4r2RHb3z5ef | Confused Customer | UAT #11 | Agent escalated to a human team member to help with the order confusion. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
XKt5LzkcDRMvMc9S8bxFpQ | Behavioral - Rush | UAT #64 | The agent failed to capture the full order initially, then escalated to a human employee for assistance. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
GF8oM66RdQuMCrUpBBgcuv | Accent - Hispanic/Latino | UAT #9 | The transcript does not show the agent directing the customer to pull forward or the next window after taking the order. |
dwCMSm8Ni64EFBb5TvMeH3 | Accent - Hispanic/Latino | Alternative side offering | The order was not fully captured as modifications included 'No Cajun Fries' but final order shows Cajun Fries instead. |
fzCC2drwX5WJ4mG67q66gN | Accent - Hispanic/Latino | Alternative side offering | Agent did not fully capture the order correctly and the order was inconsistent, and some items were missing. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
UPYUvPByrxYVE6kNRTETzT | Impatient Customer | UAT #8 | The agent failed to accurately capture the order (too many style selections) but still directed to pull forward without … |
Vb3ULd5F4cNAS4QoYRcit5 | Impatient Customer | UAT #1 | Agent did not capture the initial Ghost Pepper Sandwich order fully; it redirected the customer to pick another sandwich… |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
KQuWjzuDjj2seafpmhKKNN | Accent - Hispanic/Latino | Mid-order quantity change | The transcript shows the agent took the full order but did not direct the customer to pull forward or the next window be… |
nTqw369hozJ6qNUGJfTPMx | Accent - Hispanic/Latino | UAT #8 | Agent confirmed order but did not address error about too many selections and proceeded to close order. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
eZ6yTLtvuVJM5FYH7Piy7q | Impatient Customer | UAT #36 | The agent escalated to a human when correcting the order quantity of strawberry pies. |
noFkHxkLEw7y3zikBTzdya | Impatient Customer | UAT #9 | The agent failed to include the strawberry lemonade in the initial order confirmation before directing the customer to p… |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
EwjCrBtLcVzM2kDo4Xm7ue | High Background Noise Speaker | UAT #69 | The agent never directed the customer to pull forward or the next window and did not complete the order without escalati… |
PfebyCnuBXPugAVDrG4fdC | High Background Noise Speaker | Combo offer after sandwich | The customer was not at the drive-thru and was not properly directed to pull forward without escalation or handoff. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
HTCxSzHHV6rhjk6pCneZkZ | Confused Customer | Combo offer after sandwich | The agent initially confirmed the sandwich but then the customer added fries and drink, and the order was updated accord… |
Z8HTEap6pizLXzdEuKpEr2 | Confused Customer | UAT #1 | Agent escalated to a human employee after capturing the order but did not direct the customer forward without handoff. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
J58Yi4U6qj3fhkB75YmBhx | Accent - Hispanic/Latino | UAT #36 | The agent did not complete the order but escalated to a human employee to verify the order. |
M6NuVyVe4w9F9VtxUnvJLC | Accent - Hispanic/Latino | Combo offer after sandwich | Agent threatened to get employee help if order was incorrect, indicating escalation possibility. |
d2XYbEyVqzBELpvPhihcP3 | Accent - Hispanic/Latino | Combo offer after tenders | The agent did not fully read back the order and directed the customer to an employee at the window. |
fKkgqWpSVvxSW7Ce5GdbcN | Accent - Hispanic/Latino | Combo offer after tenders | Agent failed to confirm order with customer and escalated to human team member for help. |
iLof6KRxtaxRXRTvqKQhSr | Accent - Hispanic/Latino | Combo offer after signature chicken | The agent repeated the initial question mid-conversation, indicating confusion or failure to fully capture the order ini… |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
2WL4Gc83b4brtEJJPt7Re7 | Accent - AAVE | UAT #11 | The agent said 'Employee will help fix that,' indicating escalation to a human. |
53rJrHmmgvSnhMYre7ptib | Accent - Hispanic/Latino | Alternative side offering | Agent said order was incorrect and said 'I'll get an employee', escalating to a human, before directing to pull forward. |
A7nhVHUSUEdk3aLMVcE3yP | Accent - Hispanic/Latino | UAT #61 | Agent confirmed order but repeatedly directed customer to an employee, indicating escalation to a human. |
B64Mu9SPLBomiNnruQDBVX | Accent - Hispanic/Latino | UAT #29 | Agent failed to read back order and stated an employee would assist, indicating escalation. |
JLexbjZxHm9qgaGnJy994Q | Accent - General American | Mid-order quantity change | Agent repeatedly said 'An employee will assist you,' indicating escalation to a human. |
JvEpsQaxN8XFxnVbcNfThb | Accent - Hispanic/Latino | Combo offer after tenders | The agent escalated to a human multiple times instead of directing the customer to the next window. |
R9xyGryrKUJjTb2DSX2DH6 | Accent - Southern US | Combo offer after signature chicken | The transcript does not show the agent directing the customer to pull forward or the next window. |
W5SDL4XfrRP3gskVPZ3djw | Accent - Hispanic/Latino | UAT #10 | The agent did not direct the customer to pull forward or the next window but escalated to an employee for assistance. |
XZnRCCHeEMzeQao5QzXYAX | Accent - AAVE | Combo offer after sandwich | Agent did not direct the customer to pull forward/next window but repeatedly escalated to an employee instead. |
Z8PGksrTb7G4LiwCMsf2k4 | Accent - Hispanic/Latino | UAT #9 | Agent escalated to a human team member instead of directing customer forward after full order. |
g8TaRYP8Zy8ZFusKMm4UEA | Accent - AAVE | UAT #9 | Agent escalated to a human multiple times and did not confirm order or direct to next window alone. |
gFFKUCk8iBTEBPeB9PkoK2 | Accent - Hispanic/Latino | UAT #2 | The agent escalated to a human team member instead of directing the customer forward. |
gkr7vyP2V7yPeGujgvFoyP | Accent - Hispanic/Latino | UAT #11 | Agent escalated to a human by stating 'An employee will assist you.' |
gvQHmw3XQoFU2ZnkjSEoNC | Accent - General American | Combo offer after sandwich | The agent escalated to a human team member by saying 'An employee will assist you.' |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
7EnzSWe8PTuFvbZp7dsMc7 | Impatient Customer | UAT #64 | The agent escalated the order by saying an employee will assist the customer shortly, not directing them to the next win… |
CTANLChJxR7DgWPvKc9bAz | Impatient Customer | UAT #69 | The agent never directed the customer to pull forward or go to the next window after taking the full order. |
U5kdXS4rA39txMJm9EFHmy | Impatient Customer | UAT #11 | Agent escalated to a human employee to fix the order issue and assist the customer. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
8d7fk8AU2CXDbzMqtVo9FE | Behavioral - Complex-Order | UAT #64 | The agent escalated to a human instead of completing the order and directing the customer forward. |
HVVBFpgQhoHzjd4cdvvMQA | Behavioral - Complex-Order | UAT #64 | The agent escalated to a human employee to assist with order review before the customer could proceed. |
MYjdYwsHiwLeKSqoWpPK2g | Behavioral - Complex-Order | UAT #64 | Agent asked customer to check screen and mentioned employee assistance before directing to pull forward, indicating esca… |
UzEvTtBEDg5TwU8hfpcfdL | Behavioral - Complex-Order | UAT #64 | The agent did not read back the full order and escalated by saying 'An employee will assist you.' |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
NGwDxvYQtBMZSPq5K4hFxq | Accent - AAVE | Combo offer after signature chicken | The agent did not take the full order or direct the customer to pull forward or next window. |
RVgEWScefyQcC4AH8VHTKE | Accent - Hispanic/Latino | Combo offer after tenders | The agent did not process the order or direct the customer forward; no order completion or transfer occurred. |
TCXD4TRozAiahoy6hyx2EK | Accent - General American | UAT #42 | The agent did not confirm the full order or direct the customer to pull forward; order completion is unclear. |
bwtuinmkBbwyf6DxEtNTdm | Accent - AAVE | Combo offer after signature chicken | Order was repeated but not fully confirmed or completed, and no direction to pull forward or next window given. |
jSFonNDG82dntAiCXtKgHt | Accent - General American | UAT #9 | The agent did not capture the order or direct the customer to pull forward; the conversation ends without completion. |
nx2k4PMecNvriAMitWVswH | Behavioral - Mind-Changer | UAT #42 | The agent never completed the order or directed the customer to pull forward or the next window. |
Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
25DxMcuZszZLs94D4izC8c | Behavioral - Rush | UAT #29 | Agent did not read back the full order and escalated by saying an employee will assist. |
2UuUTSrsH3NpXjq2WyYBPN | Confused Customer | UAT #9 | Agent could not verify order alone and escalated to human for order verification. |
2VnSDr8hjGzG52bK7GbNus | Accent - Hispanic/Latino | UAT #1 | The agent did not direct the customer to pull forward or the next window, instead said an employee will assist you (esca… |
39q2xhhVwTd37yWw5RbMYb | Behavioral - Complex-Order | UAT #2 | Agent escalated to an employee for order verification, not completing order alone. |
3HKNLxBr55dwtpaT4rU67M | Accent - AAVE | UAT #9 | Agent said 'I can get an employee to help you verify it,' indicating escalation to a human team member. |
3Q7JaCgUXJnvS38LiLtRY9 | Impatient Customer | UAT #29 | The order was not fully captured due to a discrepancy in the number of Sprites, and the agent repeated the order incorre… |
5E94Br2Xyvxxd79NaMEa6F | Behavioral - Complex-Order | UAT #2 | Agent failed to read back the order and told the customer to get an employee if it looked wrong. |
5Yv8CvVdWRV2s5BvkT5mB4 | Accent - General American | UAT #1 | The agent said 'An employee will assist you,' indicating escalation to a human team member before directing to the windo… |
5YwXgKJS93LD3qDdUYQeQG | Confused Customer | Combo offer after tenders | Agent escalated to a human after customer was unsure about order confirmation. |
5czNDH2DsaV6ERMkiBFZbg | Impatient Customer | UAT #1 | Agent escalated the order by saying 'Let me get an employee to help you out.' |
5qwMkniwwoBXDjFqZjbDH5 | Impatient Customer | Combo offer after sandwich | The agent repeatedly stated an employee would assist, thus escalating to a human and not completing order alone. |
6JnfFnqBvcaH6B8677qnJb | High Background Noise Speaker | UAT #61 | The agent escalated the conversation to an employee multiple times instead of directing the customer forward. |
6dY6NBjup6EyayNDD5T4Af | Impatient Customer | UAT #42 | Agent said, 'Let me get an employee to help you out,' indicating escalation to a human team member. |
6gv7FeZMXry3Qd2XyoGtzn | Impatient Customer | UAT #10 | Agent failed to read back the full order and only instructed the customer to pull forward. |
74zFvHx89ADjtc24t8EbJ2 | Confused Customer | UAT #8 | The agent escalated to a human team member due to customer's difficulty with the screen and lack of readback capability. |
787rPLntnU4XCHHaFYiHKM | Accent - Louisiana Creole | UAT #69 | Agent asked an employee to help with order verification, escalating to a human team member. |
7UnjnrisqzeUZeij2Gowka | High Background Noise Speaker | UAT #10 | The agent escalated to a human by stating an employee will assist the customer instead of directing them to the next win… |
7eWv2tggoLKZdtGYbhx5Lt | Accent - Louisiana Creole | UAT #42 | The transcript does not show the agent directing the customer to pull forward or next window after taking the full order… |
7nU7FMtTRuZeUfgJo4HpBX | Low Volume Speaker | UAT #9 | Agent confirmed order and told customer to pull forward but then handed off to a human employee at the window. |
8SBCcweYQBnXYEfPTkzx5V | Accent - Southern US | UAT #29 | Agent escalated to a human team member to assist after user asked for order to be read back. |
Showing 20 of 179. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
26rcRhDvDR9yBHwNwMarLC | Behavioral - Rush | UAT #2 | The order had validation issues and was not fully captured; the agent did not complete the order confirmation correctly. |
3SpUmxFtWkNmfrVGy4ugMc | Confused Customer | Combo offer after sandwich | Agent did not complete order or direct customer to next window; escalated to human multiple times. |
4SnkoqfEPqHkBSxvE8jfdA | Confused Customer | Combo offer after signature chicken | Agent escalated to a human by stating 'an employee will help confirm your order'. |
4vmACyDmpNjwJ3WeGfyT8W | Confused Customer | UAT #8 | Agent escalated to a human employee instead of directing customer to the next window. |
52B7YGvu3r5CoRnRQNn7X2 | Behavioral - Mind-Changer | Mid-order quantity change | Agent confirmed order but then said 'An employee will assist you,' indicating escalation to a human. |
6Tc9Pduphhy49fR6DA8RZ9 | Confused Customer | UAT #9 | The agent did not complete the order fully by themselves and directed the customer to an employee to verify the order at… |
9drNFPezUgA3pXgFBU22Dm | Behavioral - Rush | UAT #61 | The agent said an employee will assist the customer because the customer cannot see the screen, indicating escalation. |
Ew4jR2jgJGfKHhbaKndg3q | Confused Customer | Combo offer after tenders | The agent confirmed the order but repeatedly directed the customer that an employee would assist, not to pull forward or… |
G8G6XamTqPj6QktrsNs6PH | Behavioral - Rush | UAT #29 | The order quantities and items were inconsistent and the agent did not confirm the full order accurately before telling … |
GAAadyc8EMYMBm2tx8Eq7L | Behavioral - Mind-Changer | UAT #8 | Agent failed to confirm the final order and escalated to a human employee for assistance. |
JpN8UucXq4jZZrkyoeRBZe | Confused Customer | UAT #10 | The agent escalated to a human team member for order verification instead of directing the customer forward. |
KSMX6ipCiBwz5Hcw9LrPc9 | Behavioral - Complex-Order | UAT #64 | Agent did not read back the full order and escalated to an employee to assist at the window. |
KU3yFrDNiWXiSisam8ribC | Confused Customer | UAT #42 | The agent confirmed an employee will assist the customer, indicating escalation and not completing the order fully by th… |
MSzCa2pQ7TjN6TqWRwns9r | Behavioral - Complex-Order | UAT #42 | Agent escalated to an employee after order confirmation and did not direct customer to the next window alone. |
Rr97Q6CpcqyVVJHLaGbT6i | Behavioral - Rush | UAT #8 | Agent deferred order verification to an employee, indicating an escalation to a human team member. |
SWwgoj5WiAfxPExPhJzGHL | Behavioral - Mind-Changer | UAT #69 | Agent escalated to a human employee instead of directing the customer to the next window after taking the full order. |
TVd5Zd6PX9rJ4Ur6JBjNkH | Behavioral - Complex-Order | UAT #2 | Agent couldn't read back the order and suggested employee help, indicating incomplete order handling by the agent. |
VgtP3jki2aueVB5Ue7Zzqc | Confused Customer | UAT #1 | Agent escalated to a human by saying 'Employee will assist you' and instructing customer to wait for assistance. |
XBZqEjfZT7hDDYe2vMihAF | Confused Customer | UAT #2 | The agent stated an employee would assist the customer, indicating escalation to a human team member. |
YFvGEBuddEVN6bgrAV7Hqr | Confused Customer | Alternative side offering | The transcript shows the agent took the full order but never directed the customer to pull forward or the next window, a… |
Showing 20 of 28. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
26aQqRjiNVAa9fLmcBSQv6 | Behavioral - Rush | UAT #10 | Agent escalated to a human to help confirm the order, failing to complete it fully. |
2fwqYYbv6kNoNoxQ6y8QbT | Standard Customer | UAT #61 | Agent confirmed order but escalated to a human instead of directing customer to next window. |
2ws4Q5CkdeJveK2XSjWVRy | Behavioral - Complex-Order | UAT #61 | The agent did not read back the order and mentioned an employee would assist if needed, indicating a handoff or escalati… |
3tKQRrLm8ybBTf9FXugAPC | Behavioral - Too-Talkative | UAT #1 | The agent ordered a Spicy Chicken Sandwich instead of the Ghost Pepper Sandwich the customer requested, so the order was… |
3y9pLtRVwbCnZM6j7DsRvr | High Background Noise Speaker | UAT #10 | Agent escalated to a human employee instead of completing order and directing customer. |
44XzyUDLxvjK3MgRrdzQQa | Behavioral - Mind-Changer | UAT #29 | Agent escalated to a human team member multiple times and did not send the customer forward without escalation. |
4ZYPMs7Y2EKcT3qXDmuo7Q | Behavioral - Mind-Changer | UAT #42 | Agent couldn't read back the full order and suggested employee help, indicating incomplete autonomous handling. |
4qS8JktEibVCcXqFZrmbAd | Interruptive Speaker | UAT #61 | The agent repeatedly deferred to an employee to assist the customer, indicating escalation to a human member. |
5rAFv4iU94tVmYnBZC5GJt | Accent - Southern US | UAT #2 | Agent escalated the conversation to a human team member instead of directing customer forward. |
5rDHgtRtAjRwerx2JxdpCB | Accent - Southern US | UAT #29 | Agent confirmed order but only entered quantity 1 instead of 12-piece family meal and did not handle full quantity as pe… |
6SidWtNFX9dppnNt9HqXEU | Behavioral - Complex-Order | UAT #10 | The agent could not read back the order and said an employee will assist the customer, indicating escalation to a human. |
6ayoJPdJdWkUi6kxwk78Gy | Accent - General American | UAT #36 | Agent did not read back order and referred customer to an employee, indicating escalation. |
7PqsQy33jkUtLJLdXrUejB | Confused Customer | UAT #36 | Agent did not direct customer to pull forward or next window; instead, escalated to employee assistance repeatedly. |
9AmPAFFF77Tgsj9AUHKiwH | Accent - Hispanic/Latino | Combo offer after tenders | Agent confirmed the order but the items shown included a combo dinner, not just tenders alone. |
9NMhouX5wgaxSWwMXyctgD | Interruptive Speaker | Combo offer after sandwich | Agent could not read back the order and deferred to employee assistance, indicating escalation. |
9oKMtxV4HFLpPzhGufMYCK | Impatient Customer | UAT #11 | The agent escalated to a human team member for assistance multiple times instead of completing the order. |
A3NoDbhCqKc6gQAMUkBUfj | Impatient Customer | UAT #36 | Agent escalated to a human multiple times instead of directing customer to next window. |
AfMPfPAxKpZfLmPK2mVHzL | High Background Noise Speaker | UAT #11 | Agent escalated to a human employee instead of completing the order and directing customer forward. |
AoaPp5jeYbGw4XSGvyAD8u | Accent - Hispanic/Latino | Combo offer after sandwich | The agent escalated to a human team member instead of directing the customer to pull forward or the next window. |
CEQ3nXsXs8jgaSSYaHT6Vf | Behavioral - Complex-Order | UAT #29 | Agent failed to confirm the full two-piece order correctly and denied reading back, implying incomplete capture and pote… |
Showing 20 of 91. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
2K2Go7mxi3nnp9cxjCZYRb | Behavioral - Rush | Combo offer after tenders | The agent could not read back the order and mentioned an employee would assist if it looked wrong, indicating escalation… |
3SaGtVLsRP8kTrk5LSgoJx | High Background Noise Speaker | UAT #10 | The agent escalated to a human team member instead of directing the customer to pull forward. |
3aS6hwCpzw5osg89KYtiq3 | Interruptive Speaker | Combo offer after tenders | Agent escalated to a human team member instead of directing the customer forward after taking the order. |
4FR4jwHRfqUvqgAJrF2XPT | Impatient Customer | UAT #64 | Agent failed to confirm order details verbally and ended conversation without directing customer forward. |
4KZxjEvba4FHRzyuXNfA8a | Behavioral - Rush | Alternative side offering | Agent was unable to read back the order and suggested human help, indicating escalation. |
4kVCfEQwarxg8eDWrrSesR | Behavioral - Rush | UAT #2 | The final order summary only shows a single Classic Wrap, missing the full Wraps Bundle and lemonade, indicating incompl… |
4qoqeRvqXxvXaXfaGYu72w | High Background Noise Speaker | UAT #36 | Agent escalated to a human employee to verify the order instead of completing it themself. |
57ymjVhh5DnzSDSZBwUFjK | Accent - Southern US | UAT #2 | Agent did not capture the full order; only the classic wrap combo is in the order, missing blackened tender wrap and man… |
5fcyfbcdGdE9F9j46pEUCX | High Background Noise Speaker | UAT #64 | Agent confirmed order but said 'An employee will assist you,' indicating escalation to a human. |
6mVEWpDDnpM8KL5yk6NWLJ | High Background Noise Speaker | Alternative side offering | Agent escalated to an employee instead of completing order and directing customer forward. |
6uAFrjmVpsJTtE4xgyWtAt | High Background Noise Speaker | UAT #9 | Agent escalated to a human team member to assist the customer. |
75NkfSQrS7ozRZzwQk4myT | Confused Customer | UAT #64 | Agent escalated to a human employee rather than completing order and directing customer forward. |
7riuJnj9EsfUiUUXDHXAdH | Accent - AAVE | UAT #8 | Agent rescinded initial order choice and repeated sauces without completing the new order before sending customer forwar… |
8trjuCiRv2XtYqijYbkdmi | Impatient Customer | UAT #42 | The agent escalated to a human employee instead of completing the order and directing the customer forward. |
8uDW5AWTieqisEcbnRj6Q5 | Behavioral - Complex-Order | Mid-order quantity change | Agent escalated to a human team member after failing to read back the full order accurately. |
9YRkLvHYRBJFngarQgcUUq | Accent - Hispanic/Latino | Alternative side offering | The agent escalated to a human by saying 'An employee will assist you.' after order confirmation. |
9a4D9BBy4Bcg7XP9zdy5zR | Super Fast Speaker | UAT #2 | The agent escalated the conversation to an employee instead of directing the customer to pull forward. |
ANdwgLBQEtDV5bohdghRds | Interruptive Speaker | Alternative side offering | Agent escalated to a human team member instead of completing the order and directing the customer forward. |
CQzWGhLLUeX6aDbEc8khxc | Interruptive Speaker | UAT #9 | Agent escalated to a human for order confirmation and did not send customer forward directly. |
Cw39cQ8Hw8FM6qGuwn9cAu | Behavioral - Complex-Order | UAT #64 | The agent escalated to a human team member instead of directing the customer to pull forward after taking the full order… |
Showing 20 of 91. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
38rcejCoGuJhnCxogWSQyZ | Impatient Customer | UAT #69 | Agent escalated the conversation to a human employee and did not complete the order themselves. |
3Gk8orKbzA5BV2n5zCviSW | Confused Customer | UAT #9 | Agent said 'They'll help with the rest there,' implying escalation to a human team member at the window. |
4hCuhCgrZHoJieawUtC2Qj | Accent - Louisiana Creole | UAT #61 | The agent repeatedly indicated an employee would assist, escalating instead of directing the customer forward. |
5iZvibFE8hdVKSEAJS75Kh | Accent - Hispanic/Latino | UAT #8 | Agent escalated the order to a human team member instead of sending the customer forward themselves. |
6eg7w6t6PtehbHVa2rDS5G | Confused Customer | UAT #42 | Agent completed order but escalated to a human employee instead of directing to next window. |
6zFxG2KaR9mAxYsCQ5XV7H | Interruptive Speaker | Combo offer after tenders | Agent escalated to a human employee instead of directing customer forward without handoff. |
75YQVG2BwKwBFdRs2BwGd5 | Behavioral - Mind-Changer | UAT #69 | The agent failed to complete the order and did not direct the customer to pull forward or the next window. |
7nwTK5uxnM3muZmRZvbdsp | Behavioral - Mind-Changer | Combo offer after sandwich | Agent escalated to a human by saying 'An employee will assist you' after taking the full order. |
8YVTFJAS5xUxQBRSBpCTZj | Confused Customer | UAT #64 | The agent escalated to a human employee to verify the order and assist the customer, not directing them to the next wind… |
ADJAea7YnVHnv2oHsuGmm8 | High Background Noise Speaker | Alternative side offering | The agent escalated to a human by saying an employee will assist you after order confirmation. |
Asx9YUMgKSYqkTy4hLk3jc | Confused Customer | Combo offer after signature chicken | Agent escalated to a human employee to verify the order, not completing fully independently. |
BCYd2taWD9nPWtxYyGhcPq | Low Volume Speaker | UAT #1 | Agent escalated to a human after taking the order, not sending customer forward directly. |
BN39pSHnBPgwCaeiWwTxsG | Accent - AAVE | UAT #2 | The agent failed to capture the full order correctly; the final confirmed order only includes the frozen lemonade, not t… |
CBEEirBHpsjP2eU2U3o93h | High Background Noise Speaker | UAT #2 | Agent did not read back the full order and escalated to an employee for help at the window. |
CCyEdngGiZUikhXU2yiwk5 | Impatient Customer | Combo offer after sandwich | The agent repeatedly failed to complete the order and continuously escalated to a human employee multiple times. |
CjvFS7wBFgM3xmgNxJm3V6 | High Background Noise Speaker | UAT #1 | The agent handed off the order to an employee instead of directing the customer to pull forward. |
Cpins8avsExNfCLZG5RBiN | Accent - Louisiana Creole | Combo offer after sandwich | Agent handed off the order to an employee instead of directing customer to pull forward. |
DBGwtbfXs88tk4sYyaz2sS | Behavioral - Too-Talkative | Combo offer after sandwich | Agent asked 'Is your second side dish?' indicating the order was not fully captured before directing to the window. |
DSvQwWuHREdrCAs3iyktK8 | Accent - Southern US | UAT #11 | The agent said an employee will assist, indicating escalation and no direct direction to pull forward by the agent. |
DzPQcP7FUKjdcZJVWEA2mD | Behavioral - Rush | UAT #11 | The agent said an employee will assist, indicating escalation to a human team member. |
Showing 20 of 93. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
3EpWjrBh2776hkqoaueN6b | High Background Noise Speaker | UAT #29 | The agent failed to read back the full order and escalated to a human employee to assist the customer. |
4STd2TwmSrDXXvQBLsUt69 | High Background Noise Speaker | UAT #1 | The agent escalated by saying 'An employee will assist you' instead of directing to next window. |
4TCpZJ27NnzaAhPjNjb4DF | Low Volume Speaker | Alternative side offering | Agent escalated to a human after failing to read back the full order to the customer. |
5PAw7bbX9yoDumLr86tYJg | Standard Customer | UAT #2 | Agent escalated to a human team member instead of directing the customer forward. |
8CdBxbi3ZDZuMQP6Hn2vRF | Standard Customer | Alternative side offering | The agent escalated to an employee without directing the customer to pull forward or the next window. |
8SAgx7VazhuawYPjQJYZNg | Interruptive Speaker | UAT #10 | The agent confirmed the order but did not explicitly direct the customer to pull forward or proceed to the next window. |
8pCwf9SJkhKKA8K8vihPxE | Low Volume Speaker | UAT #61 | Agent escalated to a human employee instead of directing the customer forward after taking the order. |
AjXYDUMMFsomxtPpVUciRA | Standard Customer | Combo offer after signature chicken | The agent escalated to a human by stating an employee will assist the customer, not completing the order alone. |
CLAK7gg5BC88vwXvdasykw | High Background Noise Speaker | UAT #42 | The agent escalated to a human team member by stating 'An employee will assist you.' |
EPBxn4wouWkdhJCHBGbDsb | High Background Noise Speaker | UAT #69 | The agent escalated to a human team member instead of directing to next window after full order capture. |
Ec2mvkEkd3V9qyv6EnBC4u | High Background Noise Speaker | UAT #36 | Agent escalated to a human team member instead of directing customer forward after order was taken. |
EmXP4qEJxCx7529ece5d5N | Super Fast Speaker | UAT #29 | Agent asked about drinks after initial order but never confirmed the full order correctly (quantity mismatch) before sen… |
FTyhXQuvs8NaSroBM9xt8V | High Background Noise Speaker | Combo offer after tenders | The agent escalated the order to a human team member and did not complete or direct the customer forward. |
FWqaWFgptv7zmAa8uknYoE | High Background Noise Speaker | UAT #64 | The agent escalated the interaction by saying 'An employee will assist you,' indicating a handoff to a human team member… |
FkcmvU7ndGu9jnni7DeeEY | Impatient Customer | UAT #36 | Agent escalated to a human after order confirmation, not directing customer to next window. |
HukJQEqbbkSxkGHP3bRmQP | Standard Customer | Combo offer after sandwich | The agent escalated to a human when the customer requested order read-back. |
LDXtaKxAjARmsm7m4Brnsw | Impatient Customer | UAT #10 | The agent did not read back the full order and told the customer to confirm with an employee at the window, indicating e… |
LU9oMb6vCre8kGo25z9gfQ | High Background Noise Speaker | UAT #9 | Agent did not direct the customer to pull forward; handed off to a human instead. |
MX74PsHpqNx2vyyGCLEuEb | Impatient Customer | Combo offer after signature chicken | The agent escalated the order multiple times to an employee/human team member. |
R47RU2VwngJLKkCDRb3Fhb | Standard Customer | UAT #64 | Agent did not complete the order themselves and indicated that an employee could help with unavailable item. |
Showing 20 of 38. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
4n5v4PQKe6ziKEBXkUkvxj | Impatient Customer | UAT #2 | Agent repeated the greeting and did not capture or complete the order, nor sent customer forward. |
7q9nqzAFpSwUUuNfn67y2N | Behavioral - Too-Talkative | Combo offer after signature chicken | The transcript shows the order but lacks confirmation and direction to pull forward, no indication of order completion o… |
9EacVF6ozCj95ApsQg5od8 | Accent - AAVE | Combo offer after signature chicken | The agent took the order but never directed the customer to pull forward or the next window. |
A4xoyVY5yaDPA5NQtN5cuY | Low Volume Speaker | UAT #2 | The agent repeated the initial greeting and did not confirm or complete the order, nor direct the customer forward. |
CYjcTbanorT2n9f3MYRhAP | Accent - General American | UAT #2 | The agent did not confirm the full order or direct the customer to pull forward; conversation is incomplete. |
CkzJEpZNG8xpzd4ZmdZWPR | Low Volume Speaker | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window; the order was not confirmed complete. |
DZABCVo4CWd5LadwqACPHb | Accent - General American | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window in the transcript provided. |
F9uCPCAN5hhjDjzVzjnfiW | Accent - Southern US | Combo offer after signature chicken | Agent did not complete order or direct to next window; escalated to human employee. |
FMGRr65XWLtreWbifwUhRo | Standard Customer | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window; no completion of order or direction given. |
GYYz2LWKEV9iFimg3m9Reu | Accent - Southern US | Combo offer after signature chicken | The agent never directed the customer to pull forward or the next window after taking the order. |
KUPkqXovP5NHMnb5JWrccC | Behavioral - Complex-Order | Combo offer after signature chicken | The transcript ends before the agent directs the customer to pull forward or the next window, and no order completion is… |
NRnAQqyW8ZgYZV8wepuA3v | Behavioral - Too-Talkative | Combo offer after signature chicken | Agent never confirmed full order completion or directed customer to pull forward; no indication order was finalized with… |
Qy5yL5iUwh4yRWp4MhqXaw | Interruptive Speaker | Combo offer after signature chicken | The agent escalated to a human by saying an employee will assist the customer. |
SEJ3Zib2HNrghQNaJeHxma | Super Fast Speaker | Combo offer after signature chicken | Agent escalated the order to a large combo, modifying original item, indicating partial capture and shift. |
UhSitR9h3mJKLTANURCjog | Super Fast Speaker | Combo offer after signature chicken | The agent escalated to a human team member instead of completing the order and directing forward. |
WGoeC4R2Yf9ZsSdPGo4VVU | Confused Customer | Combo offer after signature chicken | Order was not fully confirmed or completed; no direction given to pull forward or next window. |
YuNoXdLUnpiRZfG3X7caxG | Standard Customer | Combo offer after signature chicken | Agent failed to read back the order and suggested employee help, indicating escalation to a human. |
bg8hi6NJD6Wod2hr9DZ3C8 | Accent - Louisiana Creole | Combo offer after signature chicken | The conversation shows order capture but lacks agent directing customer to pull forward or next window. |
ccWJNqsxw6iwrtBr8PZxRr | Accent - Hispanic/Latino | Combo offer after signature chicken | The agent did not direct the customer to pull forward or to the next window after completing the order. |
gxPsR3qibBMyBivqrym3FB | Accent - General American | Combo offer after signature chicken | The transcript contains no agent confirmation or direction to pull forward/next window, only the customer's repeated ord… |
Showing 20 of 30. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
3a2gaMEbNEh9aZAh9FeFPY | Accent - General American | UAT #9 | The agent did not complete the order or direct the customer to pull forward or the next window. |
4swvPJzxBjPXYb7nUV6Eyn | Low Volume Speaker | UAT #9 | Agent never confirmed the full order or directed the customer to pull forward or the next window. |
5m5XkKyc3AYjffQPrDN6Sy | Confused Customer | UAT #42 | The agent did not confirm the full order or direct the customer to pull forward; no completion or forwarding stated. |
69kJV6uSxXTPnPkgDdd6GY | Accent - AAVE | UAT #64 | The agent did not complete the full order or direct the customer to pull forward or the next window. |
6fpYQYTsQ7DkiofBDYvbtD | Confused Customer | Combo offer after signature chicken | The agent did not direct the customer to pull forward or the next window after taking the full order. |
7JCtcEbn7FACdc9Agiw42k | Low Volume Speaker | UAT #1 | The agent never directed the customer to pull forward or the next window after taking the order. |
7dd6WzTQjF7b8tAi8cvhzJ | Impatient Customer | UAT #42 | The agent did not capture the order or direct the customer forward; the order was repeated by customer only. |
862ckZW28jBRbtrwmZkpmx | Behavioral - Too-Talkative | UAT #36 | The transcript shows the order was partially captured, but the agent did not explicitly direct the customer to pull forw… |
BCfBYeqYsbMyJg3pWsrJ89 | Accent - Hispanic/Latino | Combo offer after tenders | Agent did not confirm the order or direct customer forward; only repeated welcome message, no order completion. |
CLBggiuKwfSjienmPt8pnK | Accent - Hispanic/Latino | UAT #64 | The agent did not confirm the full order or direct the customer to pull forward; the order was incomplete. |
DSPqeXmv6KWEigMqQ7t5Bq | Accent - Louisiana Creole | Combo offer after tenders | Agent did not capture the order or direct the customer to pull forward; no order completion evident. |
DdxdbJEGX6S2yT6gZiDuc8 | Behavioral - Too-Talkative | UAT #10 | The transcript shows no agent response capturing the full order or directing the customer forward. |
ECeUy3TwPvitWfLys5EwdB | Behavioral - Rush | UAT #10 | The agent did not capture the full order or direct the customer to pull forward; conversation is incomplete and no order… |
FqDhExUm8TWjZhiBK9wY9S | Accent - AAVE | UAT #8 | Agent never directed the customer to pull forward or to the next window after taking the full order. |
GMZ4knYQruUS9mi2rxSyo2 | Interruptive Speaker | Combo offer after signature chicken | The transcript ends without the agent directing the customer to pull forward or the next window. |
HuviC9276Rbc3EnzeT2rBv | Impatient Customer | Combo offer after signature chicken | The transcript never shows the agent directing the customer to pull forward or to the next window, nor confirming comple… |
JgBaap3EEsUAqDQ7cHZNxz | High Background Noise Speaker | UAT #10 | The agent did not fully capture the order or direct the customer to pull forward or the next window, nor was there any i… |
Jgg67mgP9gmEGLYAwc9QxR | Accent - AAVE | UAT #10 | The agent never confirmed the full order or directed the customer to pull forward without escalation or handing off. |
KCWzNpifNNY2D486CctkHa | Standard Customer | UAT #61 | The agent did not direct the customer to pull forward or the next window after taking the order. |
KFcxdFjAXpWQN9FsZJYRif | Behavioral - Mind-Changer | UAT #61 | The transcript ends before the agent directs the customer to pull forward or the next window. |
Showing 20 of 50. Failed conversations only.
| Simulation | Persona | Test case | Judge note |
|---|---|---|---|
3TvYfiWEwZdeLKE7DtXUBf | Behavioral - Complex-Order | UAT #64 | The agent did not fully capture the order and handed off to an employee twice for assistance. |
62iXYmGhkA32fKVEVWXeCm | Behavioral - Complex-Order | UAT #64 | Agent repeatedly handed off to an employee and failed to confirm the full order or total, escalating to a human. |
DuF6RELd754Cbi67UUGskx | Behavioral - Complex-Order | UAT #64 | The agent did not read back the full order and escalated to another employee before directing the customer to pull forwa… |
Ehyq6SYu9t7AfNkUNBUZuB | Behavioral - Complex-Order | UAT #64 | Agent failed to read back the full order and escalated to an employee for assistance. |
TZQkGpgG2xyizhR5aC5jue | Behavioral - Complex-Order | UAT #64 | Agent did not complete order themselves and instead said 'An employee will assist you,' indicating a handoff. |
Failed conversations only.
| Batch | Completion | Passed |
|---|---|---|
| Simulation Run 2026-08-18 23:27 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 23:19 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 22:13 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 22:00 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 17:12 | 100.0% | 16/16 |
| Simulation Run 2026-08-18 16:53 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 16:41 | 100.0% | 17/17 |
| Simulation Run 2026-08-18 05:01 | 100.0% | 34/34 |
| Simulation Run 2026-08-17 23:46 | 100.0% | 10/10 |
| Simulation Run 2026-08-17 23:31 | 100.0% | 17/17 |
| Simulation Run 2026-08-17 02:54 | 100.0% | 17/17 |
| Simulation Run 2026-08-17 02:14 | 100.0% | 17/17 |
| Simulation Run 2026-08-13 21:47 | 100.0% | 24/24 |
| Sandbox (Main) | 97.0% | 32/33 |
| Simulation Run 2026-08-18 23:07 | 94.1% | 16/17 |
| Simulation Run 2026-08-17 03:41 | 94.1% | 16/17 |
| Simulation Run 2026-08-17 00:35 | 94.1% | 16/17 |
| Simulation Run 2026-08-17 00:34 | 94.1% | 16/17 |
| Simulation Run 2026-08-16 23:02 | 94.1% | 16/17 |
| Simulation Run 2026-08-18 22:29 | 93.8% | 15/16 |
| Simulation Run 2026-08-18 17:48 | 92.3% | 12/13 |
| Simulation Run 2026-08-18 01:29 | 90.6% | 29/32 |
| Simulation Run 2026-08-18 22:57 | 88.2% | 15/17 |
| Simulation Run 2026-08-18 02:10 | 88.2% | 15/17 |
| Simulation Run 2026-08-17 01:55 | 88.2% | 15/17 |
| Simulation Run 2026-08-17 01:45 | 88.2% | 15/17 |
| Simulation Run 2026-08-16 23:53 | 88.2% | 15/17 |
| Staging (larger sample size) | 85.3% | 29/34 |
| Simulation Run 2026-07-20 23:03 (scratch) | 83.5% | 71/85 |
| Simulation Run 2026-08-16 23:10 | 82.4% | 14/17 |
| Simulation Run 2026-08-13 22:42 | 80.0% | 16/20 |
| Test new prompt (Aug) | 68.4% | 13/19 |
| RBI Bakeoff | 67.1% | 365/544 |
| Simulation Run 2026-07-20 23:58 (scratch) | 67.1% | 57/85 |
| RBI Bakeoff 2026-07-24 | 66.5% | 181/272 |
| RBI Bakeoff 2026-07-24 Pass 5 | 66.4% | 180/271 |
| RBI Bakeoff 2026-07-23 | 65.8% | 179/272 |
| Simulation Run 2026-07-20 22:35 (scratch) | 62.7% | 64/102 |
| Simulation Run 2026-08-16 21:10 | 58.9% | 43/73 |
| Simulation Run 2026-08-17 01:07 | 50.5% | 51/101 |
| Simulation Run 2026-08-13 22:00 | 50.0% | 5/10 |
Customer asks for Cajun Rice, an item that is not on the menu. The AI is expected to say so and recommend an alternative. Judged completion is 75.8% (122/161). Below is what the AI actually said first, classified into mutually exclusive buckets. Click a bucket to see each conversation with its transcript link.
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
9f2BZY6gVG47paQzG9AVEX | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
ApnHt4udStrx64vBCfZHUG | YES | Of course, Cajun rice isn't available. Mashed potatoes or red beans and rice instead. | 1x Red Beans & Rice; 1x 12 Piece Signature Ch |
Av6BQ8Qm2wEyCD5cGH3yDp | NO | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
Bbj4X4CCfuxxqQnBZiXhL9 | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Biscuit; 1x $5 Favorites - 2 Sides (Mashed |
EmsayGRTwtHAKjBDyGKQzV | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
FMGJHwEPDMVsWYCUAMdMFx | YES | Sorry. Cajun rice isn't available. Mashed potatoes, red beans, and rice, or Cajun fries instead | 1x Cajun Fries |
FifyczcZrhDMZFMAETiCz4 | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Spicy Chicken Sandwich Large Combo (Large |
HYvq6BMnbdedMVeQKCUCw8 | YES | Sorry. Cajun rice isn't available. Beans and rice okay instead? | 1x Red Beans & Rice |
Kvrczp74hr5BwtEUHT68N8 | NO | No problem. I heard Cajun rice. It's unavailable. Want red beans and rice instead? | 1x Red Beans & Rice |
LLzMdcDVCmG6BcXPxriZxv | YES | Sorry. Cajun rice isn't available. We have red beans and rice instead. Anything else? | 1x Red Beans & Rice; 1x Biscuit |
MdaqxZU38ZwLBkbzehK5mY | YES | Sorry. Cajun rice isn't available. Red beans and rice okay instead? | 1x Red Beans & Rice |
PzTsXLDVHzca7EpHfXkzrX | NO | Sorry. Cajun rice isn't available. We have red beans and rice instead. Anything else? | 1x Red Beans & Rice |
QHs9XRtsMdeZuKuzXEwavd | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
SBhsKMkHgZZq3A23ZuSiMr | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
SFquvszrT4hvyi3z8CXAA9 | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
UP2ncKsbCAnHJNGv5P9LzB | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
UcQtA3ncETvoF99oXM5NVG | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? Regular or large? | 1x Red Beans & Rice |
Urr2d2VPELQ2ie6KVgmhDA | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
UwJimBtJ3NaRZevc8YqWYU | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice; 1x Biscuit |
VW2CEmiE4HWvQz7muCkoaC | YES | Sorry. Cajun rice isn't available. Would you like red beans and rice instead? | 1x Red Beans & Rice |
Showing 20 of 143.
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
2aiJjKhS4jzt98JMk8ZFw4 | UNKNOWN | (no reply) | empty |
HipV5TyCyxFkm5j8ucdmT5 | UNKNOWN | (no reply) | empty |
QvoVsGfGwnxKsmwYiEwDbP | UNKNOWN | (no reply) | empty |
k5rdiY9uTxR5gpRspya2Go | NO | (no reply) | empty |
5hyNecizn9tfJQdu7E4L7r | UNKNOWN | (no reply) | empty |
69kJV6uSxXTPnPkgDdd6GY | NO | (no reply) | empty |
9JuSz3DXqWBpqKMHP5zhiF | UNKNOWN | (no reply) | empty |
AZRTxAKVQSRMx9ks5BZQNT | UNKNOWN | (no reply) | empty |
CLBggiuKwfSjienmPt8pnK | NO | (no reply) | empty |
CLhjNdhWDF8HQfVGXUAhpJ | UNKNOWN | (no reply) | empty |
Ffp4UzGupnTw8ZbWx8dNUm | UNKNOWN | (no reply) | empty |
JcSV6LXTUp9z4cdK6CBJ83 | UNKNOWN | (no reply) | empty |
Q9ZAoiKYnMpptE48xc7AkJ | UNKNOWN | (no reply) | empty |
UJ9xRmpLcRazn85yfc2S66 | UNKNOWN | (no reply) | empty |
fkf7qmyNzG7ZQxuaN35jzz | UNKNOWN | (no reply) | empty |
j2peTkJpNJFZUKtwC7SoM4 | UNKNOWN | (no reply) | empty |
oG8JXudFpkP8We2HUuSBjM | UNKNOWN | (no reply) | empty |
oHDAyCc3JKQi4ZyRvfjmEU | UNKNOWN | (no reply) | empty |
a5uAzfq9qzUJdwHGRsVv7S | UNKNOWN | (no reply) | empty |
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
5ik9KqeSZ5q6EBFuEZFE6X | YES | Let me check that for you. | 1x Cajun Fries; 1x Biscuit; 1x Cane Sweeeet T |
aBpxBKFoEnNoNUrCwcRrhL | YES | Of course, take your time. What can I get started for you? | 1x Red Beans & Rice; 1x 12 Piece Signature Ch |
friVsaj8bnB4e4eTNVVuKo | YES | One moment while I check. | 1x Red Beans & Rice |
JgFG4PQBVwfSgcFVUJWhqr | YES | Moment while I check. | 1x Spicy Chicken Sandwich Large Combo (Red Be |
cfni4H85i9EDdNmfiEtSvn | NO | Of course, take your time. What can I get started for you? | 1x Red Beans & Rice; 1x 12 Piece Signature Ch |
75NkfSQrS7ozRZzwQk4myT | NO | Moment while I check. | 1x Red Beans & Rice |
H5KQU8kJwVAxzKUTDMwYTJ | None | Of course, take your time. Let me know when you're ready. | empty |
UnKq7dAcxgVADSjBrEhht7 | None | Of course, take your time. What can I get started for you? | empty |
ZPqTCA83fzoVPUwPkE7nW8 | None | Of course, take your time. Let me know when you're ready. | empty |
62iXYmGhkA32fKVEVWXeCm | NO | Of course, take your time. What can I get started for you? | empty |
Ehyq6SYu9t7AfNkUNBUZuB | NO | Of course, take your time. What can I get started for you? | empty |
8d7fk8AU2CXDbzMqtVo9FE | NO | Of course, take your time. What can I get started for you? | 1x 12 Piece Signature Chicken Family Meal (La |
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
iq3YZ5keVVpMYJ658Sihtc | YES | Yes. We do. Anything else? | 1x Red Beans & Rice |
NrUhuZ4jqkBsHCxiSwvGA5 | NO | Yes. We still have Cajun rice. What else can I get for you? | 1x 2 Piece Signature Chicken Regular Combo (S |
VgYhUwitqqw9P598U7DTZ6 | YES | Yes. We have cinnamon apple pie. Anything else? | 1x Red Beans & Rice |
hgyuPP8KDqJJEsKMvESaU3 | YES | Yes. We do. Anything else? | 1x Red Beans & Rice |
RnH6FT3GmZ3Hv6xCNUVCrZ | None | One moment while I check. Yes. We do. What else can I get you? | empty |
HVVBFpgQhoHzjd4cdvvMQA | NO | Moment while I check. Yes. We have Cajun rice available. Anything else? | 1x 3 Piece Signature Chicken Large Combo (Spi |
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
45XWUpe4n9PRL3RRg2gNFT | YES | We have red beans and rice, regular or large. | 1x Red Beans & Rice |
dmAFv6oQ8GWM769WUYxndv | NO | I understand. You like red beans and rice instead? | 1x Red Beans & Rice |
4FR4jwHRfqUvqgAJrF2XPT | NO | I understand. You like red beans and rice instead? | 1x Red Beans & Rice |
6coQ83bZgQsS4s3GPfNkza | YES | Sorry. I didn't catch that. Did you want Cajun fries or red beans and rice? | 1x Red Beans & Rice (Rice) |
n9RuKgnd3wdG4CC5eAFCxk | YES | Sorry. Did you want red beans and rice instead? | 1x Red Beans & Rice |
| Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|
MAYfprdnawRdU4QxacycXn | NO | Of course. Regular or large? | 1x Red Beans & Rice |
fGV8TzvyGwyeNYb2i5pYPZ | YES | Sorry. I didn't catch that. What drink would you like? | 1x Red Beans & Rice |
oDLsiRp4YJiwNvRECoCVzv | NO | Hi. Welcome to Popeyes. Can I get started for you today? | empty |
XKt5LzkcDRMvMc9S8bxFpQ | NO | Is your order at the bottom of the screen on the right correct? | empty |
| Behavior | Count | Share |
|---|---|---|
| Denied, offered alternative | 143 | 75.7% |
| No reply | 19 | 10.1% |
| Stalled, never answered | 12 | 6.3% |
| False affirmation | 6 | 3.2% |
| Silent substitution | 5 | 2.6% |
| Off-topic / no answer | 4 | 2.1% |
Simulation IDs link straight to the transcript in Coval.
| Behavior | Simulation | Completion | What the AI said | Final cart |
|---|---|---|---|---|
| Silent substitution | n9RuKgnd3wdG4CC5eAFCxk | YES | Sorry. Did you want red beans and rice instead? | 1x Red Beans & Rice |
| Silent substitution | 6coQ83bZgQsS4s3GPfNkza | YES | Sorry. I didn't catch that. Did you want Cajun fries or red beans and rice? | 1x Red Beans & Rice (Rice) |
| False affirmation | HVVBFpgQhoHzjd4cdvvMQA | NO | Moment while I check. Yes. We have Cajun rice available. Anything else? | 1x 3 Piece Signature Chicken Large Combo (Spi |
| False affirmation | RnH6FT3GmZ3Hv6xCNUVCrZ | None | One moment while I check. Yes. We do. What else can I get you? | empty |
| False affirmation | hgyuPP8KDqJJEsKMvESaU3 | YES | Yes. We do. Anything else? | 1x Red Beans & Rice |
| Silent substitution | 4FR4jwHRfqUvqgAJrF2XPT | NO | I understand. You like red beans and rice instead? | 1x Red Beans & Rice |
| False affirmation | VgYhUwitqqw9P598U7DTZ6 | YES | Yes. We have cinnamon apple pie. Anything else? | 1x Red Beans & Rice |
| False affirmation | NrUhuZ4jqkBsHCxiSwvGA5 | NO | Yes. We still have Cajun rice. What else can I get for you? | 1x 2 Piece Signature Chicken Regular Combo (S |
| Silent substitution | dmAFv6oQ8GWM769WUYxndv | NO | I understand. You like red beans and rice instead? | 1x Red Beans & Rice |
| Silent substitution | 45XWUpe4n9PRL3RRg2gNFT | YES | We have red beans and rice, regular or large. | 1x Red Beans & Rice |
| False affirmation | iq3YZ5keVVpMYJ658Sihtc | YES | Yes. We do. Anything else? | 1x Red Beans & Rice |
| Off-topic / no answer | XKt5LzkcDRMvMc9S8bxFpQ | NO | Is your order at the bottom of the screen on the right correct? | empty |
| No reply | fkf7qmyNzG7ZQxuaN35jzz | UNKNOWN | (no reply) | empty |
| No reply | Q9ZAoiKYnMpptE48xc7AkJ | UNKNOWN | (no reply) | empty |
| No reply | AZRTxAKVQSRMx9ks5BZQNT | UNKNOWN | (no reply) | empty |
| No reply | 9JuSz3DXqWBpqKMHP5zhiF | UNKNOWN | (no reply) | empty |
| No reply | CLhjNdhWDF8HQfVGXUAhpJ | UNKNOWN | (no reply) | empty |
| No reply | j2peTkJpNJFZUKtwC7SoM4 | UNKNOWN | (no reply) | empty |
| Off-topic / no answer | oDLsiRp4YJiwNvRECoCVzv | NO | Hi. Welcome to Popeyes. Can I get started for you today? | empty |
| No reply | oHDAyCc3JKQi4ZyRvfjmEU | UNKNOWN | (no reply) | empty |
| No reply | 5hyNecizn9tfJQdu7E4L7r | UNKNOWN | (no reply) | empty |
| No reply | CLBggiuKwfSjienmPt8pnK | NO | (no reply) | empty |
| No reply | oG8JXudFpkP8We2HUuSBjM | UNKNOWN | (no reply) | empty |
| No reply | JcSV6LXTUp9z4cdK6CBJ83 | UNKNOWN | (no reply) | empty |
| No reply | Ffp4UzGupnTw8ZbWx8dNUm | UNKNOWN | (no reply) | empty |
| No reply | UJ9xRmpLcRazn85yfc2S66 | UNKNOWN | (no reply) | empty |
| No reply | 69kJV6uSxXTPnPkgDdd6GY | NO | (no reply) | empty |
| No reply | a5uAzfq9qzUJdwHGRsVv7S | UNKNOWN | (no reply) | empty |
| No reply | 2aiJjKhS4jzt98JMk8ZFw4 | UNKNOWN | (no reply) | empty |
| No reply | QvoVsGfGwnxKsmwYiEwDbP | UNKNOWN | (no reply) | empty |
| No reply | HipV5TyCyxFkm5j8ucdmT5 | UNKNOWN | (no reply) | empty |
| No reply | k5rdiY9uTxR5gpRspya2Go | NO | (no reply) | empty |
| Stalled, never answered | 8d7fk8AU2CXDbzMqtVo9FE | NO | Of course, take your time. What can I get started for you? | 1x 12 Piece Signature Chicken Family Meal (La |
| Stalled, never answered | Ehyq6SYu9t7AfNkUNBUZuB | NO | Of course, take your time. What can I get started for you? | empty |
| Stalled, never answered | 62iXYmGhkA32fKVEVWXeCm | NO | Of course, take your time. What can I get started for you? | empty |
| Stalled, never answered | H5KQU8kJwVAxzKUTDMwYTJ | None | Of course, take your time. Let me know when you're ready. | empty |
| Stalled, never answered | UnKq7dAcxgVADSjBrEhht7 | None | Of course, take your time. What can I get started for you? | empty |
| Stalled, never answered | ZPqTCA83fzoVPUwPkE7nW8 | None | Of course, take your time. Let me know when you're ready. | empty |
| Stalled, never answered | 75NkfSQrS7ozRZzwQk4myT | NO | Moment while I check. | 1x Red Beans & Rice |
| Off-topic / no answer | fGV8TzvyGwyeNYb2i5pYPZ | YES | Sorry. I didn't catch that. What drink would you like? | 1x Red Beans & Rice |
| Stalled, never answered | cfni4H85i9EDdNmfiEtSvn | NO | Of course, take your time. What can I get started for you? | 1x Red Beans & Rice; 1x 12 Piece Signature Ch |
| Stalled, never answered | JgFG4PQBVwfSgcFVUJWhqr | YES | Moment while I check. | 1x Spicy Chicken Sandwich Large Combo (Red Be |
| Off-topic / no answer | MAYfprdnawRdU4QxacycXn | NO | Of course. Regular or large? | 1x Red Beans & Rice |
| Stalled, never answered | 5ik9KqeSZ5q6EBFuEZFE6X | YES | Let me check that for you. | 1x Cajun Fries; 1x Biscuit; 1x Cane Sweeeet T |
| Stalled, never answered | aBpxBKFoEnNoNUrCwcRrhL | YES | Of course, take your time. What can I get started for you? | 1x Red Beans & Rice; 1x 12 Piece Signature Ch |
| Stalled, never answered | friVsaj8bnB4e4eTNVVuKo | YES | One moment while I check. | 1x Red Beans & Rice |
The Ghost Pepper Sandwich is not in the POS menu. Across 288 judged conversations the agent never carted it once; it substitutes Classic or Spicy silently about half the time, the same class of failure RBI flagged on Cajun Rice. Full findings, disclosure rates, and every non-disclosed conversation: Ghost Pepper UATs page.
Org rbi-bakeoff-deepgram, workspace default.
Top level: Runs ·
Test sets ·
RBI bakeoff test set ·
Reports
| Test case | ID | Scenario |
|---|---|---|
| Generic capability - Alternative side offering | ZZqF3btvf4ELGahb5meU8G | I'd like the 3-piece signature chicken combo, spicy. I don't want fries. What other sides |
| Generic capability - Combo offer after sandwich | MwSegbsoJ4Er6de8puhjud | Order one classic chicken sandwich by itself. Do not ask for a combo. If the agent offers |
| Generic capability - Combo offer after signature chicken | NovS6SvpaUZEn6FPsNowB8 | Order three pieces of spicy signature chicken by themselves. Do not ask for a combo. If th |
| Generic capability - Combo offer after tenders | gyeBnx8BuHbekXt9i8oFwo | Order five classic tenders by themselves. Do not ask for a combo. If the agent offers to m |
| Generic capability - Mid-order quantity change | CbsRpJUKVqjiBviRazmHVN | Start by ordering the 3-piece signature chicken combo, spicy, with mashed potatoes and a C |
| RBI connectivity smoke | kkdbhUXWYEuoqy8Pn9oyeY | Order one classic chicken sandwich, no combo. Complete the order, then stop speaking when |
| UAT #1 | 8kg9tJuQt9wgpbovbw7He5 | One Ghost Pepper Sandwich, no combo. |
| UAT #1 | JKoLD3Ya5iNLxoKeNrzput | One Ghost Pepper Sandwich, no combo. |
| UAT #10 | j6pnhcuCTnR3jo8xxNWzcF | Can I get the ghost pepper sandwich combo, swap the fries for red beans & rice, and a Dr P |
| UAT #11 | X2GRYX9HsnXcZhxqMr9W9S | I'll do the chicken sandwich by itself and a side of fries. |
| UAT #2 | ftT5fiXMKfuqbfsbcZMWRM | Give me the chicken wrap bundle with a mango lemonade. |
| UAT #29 | Ldk7ASyKLATimVuEzwQAKs | I'll do the 12-piece family meal with all dark meat, spicy, Cajun fries and cole slaw, and |
| UAT #36 | ixXXgJAQJuKZRQU4PQ87wM | I'll take the 3-piece chicken combo, spicy, add a pie - make it the strawberry - and a pre |
| UAT #42 | NYcmgkF7qQ75WeDZ3Sgrxb | Can I get the Ghost Pepper Sandwich combo? Actually, change that to the regular chicken sa |
| UAT #61 | PyWRkMc6pdfYPBB6ZsiRe6 | Order one regular bacon and cheese sandwich by itself. Do not ask for a combo or a large. |
| UAT #64 | 28x3gHvpCX6kssjtfZFLcb | Can I have a Cajun Rice please? |
| UAT #69 | VfwbVocDCm2aNGSfvakriG | Do you have nuggets? |
| UAT #8 | JKTf9UwAYRpPaUK3bnup4u | I'd like the 5-piece tenders combo, blackened and spicy, with cole slaw and a Coke. |
| UAT #9 | ahpduY88435y47NjtFoGpS | One bacon and cheese sandwich combo, large, with fries and strawberry lemonade. |
| Run | ID | Completion | Passed | Tags |
|---|---|---|---|---|
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — High Background Noise Speaker — Batch 2 | 7M86byb9g7feToqnuhmsyE | 27% | 3/11 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — High Background Noise Speaker — Batch 2 — continuation | WCsiVgo3ceG5zpwtwK7etY | 0% | 0/6 | batch-2, continuation, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — High Background Noise Speaker — Batch 3 | 4kFuzxzWeznZQkEagtFF3K | 12% | 2/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Impatient Customer — Batch 2 | nHi9cbPGWM7KbRP3fx5nXM | 29% | 5/17 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Impatient Customer — Batch 3 | a28zHrGWV6JWQT8utVQNmP | 29% | 5/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Interruptive Speaker — Batch 2 | 4Snh9RtHZxvLPb3Dp65vSf | 65% | 11/17 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Interruptive Speaker — Batch 3 | 9eaTg684vEUuqDwbrvTS5U | 76% | 13/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Low Volume Speaker — Batch 2 | aognrjb8X7ZFDH5A9y5Uwx | 76% | 13/17 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Low Volume Speaker — Batch 3 | Q5VvSi2E32wVPp5n6yiKuw | 53% | 9/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Standard Customer — Batch 2 | gd9cWeE3vUtFbgzn8PEagX | 71% | 12/17 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Standard Customer — Batch 3 | 2NkWiUDcLkAgxUa7bubL4z | 76% | 13/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Super Fast Speaker — Batch 2 | oY8uwSfBYNc4JRG2Rtix52 | 88% | 15/17 | batch-2, rbi-bakeoff, suite-a |
| RBI Bakeoff — Suite A (Audio Robustness) — Deepgram — Super Fast Speaker — Batch 3 | 4kmA9NbQe52EksczXAjDhm | 94% | 16/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — AAVE — Batch 3 | 6bQ8R5ZrKkxSQrWG4wy6RF | 76% | 13/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Accent - AAVE — Batch 2 | FEmmyxHXJ8kmB7pfmYmwZ6 | 76% | 13/17 | batch-2, rbi-bakeoff, suite-b |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Accent - General American — Batch 2 | Ri8H4fU6koUkopPEu9i5Aj | 94% | 16/17 | batch-2, rbi-bakeoff, suite-b |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Accent - Hispanic/Latino — Batch 2 | 9RdgBGfsB6SCEP7TCTYjYu | 53% | 9/17 | batch-2, rbi-bakeoff, suite-b |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Accent - Louisiana Creole — Batch 2 | Krj8Bazpcrv2xsAwXXsgyX | 82% | 14/17 | batch-2, rbi-bakeoff, suite-b |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Accent - Southern US — Batch 2 | Noe7EoFev7sgHYZDHmmrtn | 88% | 15/17 | batch-2, rbi-bakeoff, suite-b |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — General American — Batch 3 | 46idNJm5oXFofnv8Vq5Ztp | 65% | 11/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Hispanic Latino — Batch 3 | b3uXvsdbEASrVZFKG7xK9e | 47% | 8/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Louisiana Creole — Batch 3 | gttG557SyJpUXxacQxjKfT | 94% | 16/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite B (Accent Robustness) — Deepgram — Southern US — Batch 3 | e59QbffVu4PQ6VPjc743yc | 94% | 16/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Behavioral - Complex-Order — Batch 2 | czoZVyDV6kUreMYEDk2y67 | 82% | 14/17 | batch-2, rbi-bakeoff, suite-c |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Behavioral - Mind-Changer — Batch 2 | ft5UzfY4JAaVK8XV6TbgpX | 88% | 15/17 | batch-2, rbi-bakeoff, suite-c |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Behavioral - Rush — Batch 2 | FrmXHBKmG3cC2RK2ug4gqD | 82% | 14/17 | batch-2, rbi-bakeoff, suite-c |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Behavioral - Too-Talkative — Batch 2 | SpwVB3bmBer9nWSp2pFh7u | 88% | 15/17 | batch-2, rbi-bakeoff, suite-c |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Complex Order — Batch 3 | oARArF6jYsxApNvWqqowrf | 59% | 10/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Confused Customer — Batch 2 | iuT428utH3XWMiZTRQyTyg | 24% | 4/17 | batch-2, rbi-bakeoff, suite-c |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Confused Customer — Batch 3 | J3ZM628REKBN2fEKPuvvop | 18% | 3/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Mind Changer — Batch 3 | i4UKHTQ88LGXnU99R6ghi9 | 82% | 14/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Rush — Batch 3 | fZBZYBs75rVgYjvg9753Bc | 65% | 11/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff — Suite C (Behavioral) — Deepgram — Too Talkative — Batch 3 | 2h9jQ84vbZvPL3qpWLZpFw | 100% | 17/17 | batch-3, official, rbi-bakeoff |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — MUFJdYAHdBHU6UbRRu4ykM | V8aN7maJysfHYydeYDQLim | 76% | 13/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — NNMm2vhQRW5E747p7SzuNM | Nyz4DbhbfgxWdWENF49ZRR | 71% | 12/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — VSruLZYYyz3w7fkdUaheDR | SHKLEmB8TvhHV9hkQCGcGo | 76% | 13/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — VZ3mLhnQ5SgpkZeNHET8fq | LEApgktYYweX9mvqCmYnQQ | 53% | 9/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — ZGQcBTzyxoRYNUa7BSw8i4 | LXSzETgB4eMR2GQ6ajbhc5 | 76% | 13/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — Suite A — deepgram — jDPgdyKRFxxbWgCcLQJ5qS | YmsSzdanPKbqK6eHnqAE2K | 12% | 2/17 | 2026-07-23, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-23 — suite-b — deepgram — AQPvfyLYjJmuJ2AdQ3gGCi | RAwN5bPsyMnmgjJBysbGYi | 71% | 12/17 | 2026-07-23, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-23 — suite-b — deepgram — BMQEKLKDPyKpv3Bocdh5L7 | CxMzamWsyffYFvDCzcNicZ | 88% | 15/17 | 2026-07-23, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-23 — suite-b — deepgram — ETDujiUnZoQbH3rrR2HbmK | CW6UoRXpXTLg64QbbLmbxi | 88% | 15/17 | 2026-07-23, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-23 — suite-b — deepgram — LD8JybxS3R9Q9u6hiPwX8a | 2WMchnW4UKNAC8v8KWk7ji | 88% | 15/17 | 2026-07-23, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-23 — suite-b — deepgram — YZyUB5juT4fFLGbmY26m3N | 8pUxWnqAzSCctSQao2UE4d | 47% | 8/17 | 2026-07-23, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-23 — suite-c — deepgram — Cf3iFSCoqTgyqxtgXG3g28 | kRQSGx9orqJAMo7N8LD7Jk | 71% | 12/17 | 2026-07-23, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-23 — suite-c — deepgram — EXUdeySkLnjqWqtRyGwe2Z | XhLyPCofnGC9roBd9Q3kC2 | 71% | 12/17 | 2026-07-23, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-23 — suite-c — deepgram — Xw4izPULtn49u7xDtPammb | Xg3FkKqWrPeTRbrESCiZE5 | 59% | 10/17 | 2026-07-23, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-23 — suite-c — deepgram — fDd7TCj4cfa7xGUSuPRYdV | D98M8fm9KPXnbqMZfQHW24 | 18% | 3/17 | 2026-07-23, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-23 — suite-c — deepgram — fWeBJDoqgo8BfLyWcSgdFw | 8rfrDEEVkF7Hqu4V89z3Kn | 88% | 15/17 | 2026-07-23, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — MUFJdYAHdBHU6UbRRu4ykM | AGQgKu6fsxaSQXPiHupb2h | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — MUFJdYAHdBHU6UbRRu4ykM — Replacement 1 | BSntEkFpnrF9f8jFV4skMd | 65% | 11/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — NNMm2vhQRW5E747p7SzuNM | 2xC9ovyCDTCmEYRPcMLsNG | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — NNMm2vhQRW5E747p7SzuNM — Replacement 1 | LAzUuEbTFwLvboEk9PGtdW | 94% | 16/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — VSruLZYYyz3w7fkdUaheDR | if7FWrJ2gbpihbA3HtEseo | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — VSruLZYYyz3w7fkdUaheDR — Replacement 1 | br4csueRTiePPJkzPhQtiL | 65% | 11/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — VZ3mLhnQ5SgpkZeNHET8fq | AgrkSFkXQFwZkk3yenB3JY | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — VZ3mLhnQ5SgpkZeNHET8fq — Replacement 1 | h4XMEhfLrKrpBQMBxDPDX8 | 41% | 7/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — ZGQcBTzyxoRYNUa7BSw8i4 | PdRzFmGS9Re6pBPCPrcFR4 | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — ZGQcBTzyxoRYNUa7BSw8i4 — Replacement 1 | MGfN27rNC2YMWwBTpDs7nV | 76% | 13/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — jDPgdyKRFxxbWgCcLQJ5qS | NgfrdL3qsBAx6S4EeAzDAt | 0% | 0/0 | 2026-07-24, official, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 — Suite A — deepgram — jDPgdyKRFxxbWgCcLQJ5qS — Replacement 1 | Q8bo6hWGiH3mXdxT4cpxG5 | 24% | 4/17 | 2026-07-24, official, rbi-bakeoff, replacement-1, suite-a |
| RBI Bakeoff 2026-07-24 — Suite B — deepgram — AQPvfyLYjJmuJ2AdQ3gGCi | YrZy6B8iK9Npe6PKhTRx6j | 94% | 16/17 | 2026-07-24, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 — Suite B — deepgram — BMQEKLKDPyKpv3Bocdh5L7 | Pm9eq6aGTa2NtRUihCDN64 | 88% | 15/17 | 2026-07-24, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 — Suite B — deepgram — ETDujiUnZoQbH3rrR2HbmK | hDJVUxSSPv9wgTfXTDaeRT | 76% | 13/17 | 2026-07-24, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 — Suite B — deepgram — LD8JybxS3R9Q9u6hiPwX8a | Rb7Ex46mcwL8UxkbsxFZnY | 82% | 14/17 | 2026-07-24, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 — Suite B — deepgram — YZyUB5juT4fFLGbmY26m3N | hm44DhpjGCbX6UcFZtFExW | 53% | 9/17 | 2026-07-24, official, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 — Suite C — deepgram — Cf3iFSCoqTgyqxtgXG3g28 | R3mzVukjnukuikv6n5shqP | 82% | 14/17 | 2026-07-24, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 — Suite C — deepgram — EXUdeySkLnjqWqtRyGwe2Z | A4MBmeHchHDgedMVf4BHFZ | 71% | 12/17 | 2026-07-24, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 — Suite C — deepgram — Xw4izPULtn49u7xDtPammb | CfHevXMCFexuRHwUbk5TSM | 65% | 11/17 | 2026-07-24, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 — Suite C — deepgram — fDd7TCj4cfa7xGUSuPRYdV | LUWWXwGHv8YsVh6uLG4CJG | 0% | 0/17 | 2026-07-24, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 — Suite C — deepgram — fWeBJDoqgo8BfLyWcSgdFw | fUtpJ9tmtaoxPJWCb5MtFB | 88% | 15/17 | 2026-07-24, official, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — MUFJdYAHdBHU6UbRRu4ykM | 9yXvShbGcHoSjJaw94xtKT | 71% | 12/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — NNMm2vhQRW5E747p7SzuNM | NdDU346guxiWeVAJBa6BmV | 88% | 15/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — VSruLZYYyz3w7fkdUaheDR | NxdbURsGt2hJiXdoCLdyzg | 65% | 11/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — VZ3mLhnQ5SgpkZeNHET8fq | L4SAsehLwCFRZHSDK7Rjyv | 53% | 9/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — ZGQcBTzyxoRYNUa7BSw8i4 | YtPnVbBXVTJHCP3W5zwi8v | 76% | 13/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite A — deepgram — jDPgdyKRFxxbWgCcLQJ5qS | aMrZESM6ShKExcE8bStNfg | 24% | 4/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-a |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite B — deepgram — AQPvfyLYjJmuJ2AdQ3gGCi | AQuTjJYTgmb3n6a7oMsgFb | 88% | 15/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite B — deepgram — BMQEKLKDPyKpv3Bocdh5L7 | mFWog74SPpFXMsna7sSGWZ | 82% | 14/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite B — deepgram — ETDujiUnZoQbH3rrR2HbmK | cPKSMrwZmNcnuW7HoArqj9 | 94% | 16/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite B — deepgram — LD8JybxS3R9Q9u6hiPwX8a | 8x32oSqkyxcTbZCSZZLwb6 | 82% | 14/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite B — deepgram — YZyUB5juT4fFLGbmY26m3N | Xk9cdcoUFNxCsVT9A4JMou | 53% | 9/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-b |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite C — deepgram — Cf3iFSCoqTgyqxtgXG3g28 | QtebrkhHCWqiHc8xsAXthp | 76% | 13/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite C — deepgram — EXUdeySkLnjqWqtRyGwe2Z | Zk3DR5KCYAZzJrXB4HY4We | 53% | 9/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite C — deepgram — Xw4izPULtn49u7xDtPammb | 63gWdeLfaRZUogPR7nHNAH | 35% | 6/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite C — deepgram — fDd7TCj4cfa7xGUSuPRYdV | a4Bgp8Q7Nk5sDAo9JCyK57 | 25% | 4/16 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-c |
| RBI Bakeoff 2026-07-24 Pass 5 — Suite C — deepgram — fWeBJDoqgo8BfLyWcSgdFw | TWs6fVjjaEjaJw7wW7Wzbk | 94% | 16/17 | 2026-07-24, official, pass-5, rbi-bakeoff, suite-c |
| RBI v2 clean connectivity proof - deepgram | ZxPJvJTDgruo7tibGoLqut | 0% | 0/1 | canonical-v2, clean-proof, rbi-bakeoff, smoke |
| RBI v2 connectivity smoke - deepgram | 9Ms3NLzx9R9DKo7g7GTm4i | 0% | 0/1 | canonical-v2, rbi-bakeoff, smoke |
| Sandbox (Main) | fQHnjeYBosHiMu4cDNx3T9 | 97% | 32/33 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-07-09 21:45 | K3o5PjYypDGzhBUFLZkQuL | 0% | 0/0 | none |
| Simulation Run 2026-07-09 23:02 | KBcKHcLfMLGyGqrgSw8Nnt | 0% | 0/0 | none |
| Simulation Run 2026-07-20 22:35 | ZjpnTFN5vvs8qRkKiwESGm | 59% | 10/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 22:35 | VeYcxZethGV5fooRN2WVbm | 71% | 12/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 22:35 | o2zJioPTZEcnSPp2nsVCDC | 82% | 14/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 22:35 | 2qSRzcQ3WcA22sBsokwYGu | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 22:35 | W3QCYqXeq6Epuqij2ggreJ | 12% | 2/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 22:35 | PAJag5Cv2GWjMiWrc8jL2m | 65% | 11/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, suite-a, vendor-eval, voice |
| Simulation Run 2026-07-20 23:03 | 8Ywj7sBwkfetSppgpNA52x | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:03 | Ha4Q9rbL4Y6uwfjpdqLvpc | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:03 | UyBwYnCNm8RS74aC4Cr9UG | 82% | 14/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:03 | UoUcmtamYtp3XE5KXJcb6F | 53% | 9/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:03 | 8a3sNhKBZyjBnuGtrffMRn | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:58 | EnFdRDLajiEqjchoyQLKrQ | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:58 | hDEYHkmDYsz8EzifpHbari | 76% | 13/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:58 | 9Vv6kjGA3Wy9bAY9LdHqFp | 71% | 12/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:58 | 7tVZPXTFXL3GwtdNWb8WTj | 59% | 10/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-07-20 23:58 | Jmw9caxgbaJ79GMhz7Mfjn | 29% | 5/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, vendor-eval, voice |
| Simulation Run 2026-08-13 19:52 | EDPdkfKJdeNE9oaCkgL6iY | 100% | 1/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, rbi-builtin, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-13 19:58 | BmZNqx6udLhdBCHzXhWPYE | 0% | 0/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-13 21:27 | jN7Bkd7XvvmXJYEzSTG3AF | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-13 21:47 | 8eXACgipEsgguwEoruA8HW | 100% | 24/24 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-13 22:00 | hDf6nRGB5rHgiNYfDeBGyF | 50% | 5/10 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-13 22:42 | aVz23yWCqaVmazx3pSzVzu | 100% | 10/10 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-13 22:42 | QHUmTWwxBYSqG3Rb5nozzn | 60% | 6/10 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | h2pF6843QBCaTU9oYLq27a | 0% | 0/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | j2RhyQc5bz92HiJUixUrzd | 0% | 0/3 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | SEP8BPnziQ25USmpvobjJJ | 0% | 0/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | 2ib7KKwGC5jUYpPA6eN8sA | 50% | 1/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | UhxyrZMj8dh6DhdbYkynTh | 50% | 1/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | 9M2eSrRisqqSRndeS6e4KR | 83% | 5/6 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | TCRJ4obb32uuRVyE6HKZrb | 100% | 8/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | X25Rpy4f5spj6UL5mZu47H | 25% | 1/4 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | oGmCEPYvyqYMdniBNWn2na | 100% | 3/3 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | UuggE2PT3FxoW6LBgZZVK9 | 71% | 5/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | JKW3oJPiTo4JiRyZQyXK9c | 78% | 7/9 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | iRK58Yye6y4NZGvgurTNsA | 50% | 1/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | QhKnx7VYXvyctRTTFyymzD | 75% | 6/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | 2gM9TNLUxDLM7MJQEAarWu | 20% | 1/5 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | 2YPkYmFujQXpK6oC6sTJhg | 57% | 4/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:10 | HDRngJ8pqq2qsSfEGmQckb | 0% | 0/3 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, round 2 test run, vendor-eval, voice |
| Simulation Run 2026-08-16 21:47 | hGmmC5atBeP3XjcE2mtgjb | 100% | 1/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-16 23:02 | HtKcup8g2b4W46xi8pjVAa | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-16 23:10 | SgBZXUvzms2Zj8zhukYXyC | 82% | 14/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-16 23:53 | EY9JnKX6NhwRhMjbRcgHyW | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 00:34 | LAMoNwWxCwidEQwTzAR32a | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 00:35 | GKYQnYENBRfrzKj59jwmh7 | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | Av8YYvrgQHYKv9sPiwMJzH | 38% | 3/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | mrMp4qWhtdYHxXA8YTVQDC | 17% | 1/6 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | GM5iSe73y2WBHcttnAjgxr | 20% | 1/5 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | Yapr7NnAiRcUpUeV3tvxxh | 40% | 2/5 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | N5GEMkoqfxe5Pxq6Wj68Fn | 50% | 2/4 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | AxfACNUDgMAx4zXhe72bhf | 50% | 3/6 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | 6v9FcD2ibn3B7jCvNs4AsU | 43% | 3/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | HbSgeRB7rEtCobNSmpHaMp | 50% | 4/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | 4uGQwj2v2zki8bat4ZwLQQ | 17% | 1/6 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | crf99KJt5CjkLbtTcQ922n | 43% | 3/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | GtJKGVLZ6HkKTb5FHt7BQG | 57% | 4/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | dwrxgfftFLr6yVdTgdmNEV | 80% | 4/5 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | 45dtyZkQUD9GsaQZsCphnT | 75% | 3/4 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | 5HPuMJsBSv4myiik6FVhWx | 75% | 6/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | BTEW6zq9jnCrJRAPKDzfnP | 86% | 6/7 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:07 | f4u8e987qekwZh2KG7KG7J | 62% | 5/8 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:45 | FQLAr8QFsRk236kpCUTF9p | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 01:55 | iEWkNjrLXqyESud67LgM3S | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 02:14 | GSojcaWPhcFeDb5NWqjqSJ | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 02:54 | 4d73fNr2n7z2S2pQTTfLwb | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 03:41 | Wnj7VammZvtYNBEeECnTYr | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 23:31 | Re5DHFDAHWBgujFRNJmmNa | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 23:40 | 8hSQjfQb2bbPJ9fVQmkUUG | 100% | 1/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-17 23:46 | Rx4CgpBveFphJB659t46XR | 100% | 10/10 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 01:29 | cz2scittc5ufDfGCbSyWvN | 91% | 29/32 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 02:10 | eKHeGGuU5JXGtf5YEi3wx5 | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 04:44 | BosTmYiWCisTQooHBwrUuM | 100% | 2/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 04:49 | erGhyrUxC7sgUW8CAsUq7D | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 04:51 | LmnbofRuf5L6JcvRRDAi4W | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 05:01 | oTxLJ4A3vT5LUELj65TAj3 | 100% | 34/34 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 16:41 | cZ4hAHfe2fcALiYoSpXhnE | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 16:53 | KCD4dWTdpx8uq5ZX4wWqp8 | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 17:12 | iU7B7idqNBg4pEhpSrRDFM | 100% | 16/16 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 17:48 | 8hJnbX3vmR7zkdHuBDN6zC | 92% | 12/13 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:00 | TYpmX4rKHXF8wenDLqoN8f | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:13 | 8SEbnfrmb4GwxFAb2LKtMQ | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:27 | dpjpU6bGBiQ3BApY8NApVA | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:29 | QCWQk6T82LBYdX26NLSUho | 94% | 15/16 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:29 | JzuPxivfBr4TrEe5khSuQM | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 22:57 | 4JFeR3i8DPy2V5MjHkcvX5 | 88% | 15/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 23:07 | 9qW4hvExaQxPSDg3qBkii5 | 94% | 16/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 23:19 | KMto5kZdXPtzTh3LKAXLWM | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 23:27 | nVHmMrjx5nhaAnD3ySXb3L | 100% | 17/17 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Simulation Run 2026-08-18 23:57 | 4EWuowxdrH7rba9nMt6s6o | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Staging (larger sample size) | KdYwQoiDKjYyUqxXUV8Zvs | 85% | 29/34 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Accent - AAVE) | 5vNWncKXPBhEo3AubrZ3Ao | 0% | 0/2 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Accent - General American) | cSbWMynYhak2xqmFAcKv4e | 50% | 2/4 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Accent - Hispanic/Latino) | anzWsJjN8trHGMFupJCZ6W | 0% | 0/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Accent - Louisiana Creole) | 6vUmP73rWAy3nfPndsF8jL | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Accent - Southern US) | cwdZZNDELUTLXyLAvfHsMV | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Behavioral - Complex-Order) | bbnhVKZfoZm9vNM4SnRs2j | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Behavioral - Mind-Changer) | aDCov4mqzwi5VkrW7NhygN | 0% | 0/1 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Behavioral - Rush) | AtghxVXHBZ6JyxeBynMZBx | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Behavioral - Too-Talkative) | 9HavQkyjoRh5rdjUVjD8h3 | 100% | 11/11 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Confused Customer) | fM8gRQADLo54SGfLPYTZs3 | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, High Background Noise Speaker) | NKaBZmFjW7zdD6Xnjdgotw | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Impatient Customer) | 5bhTAs8memtESJSEoGtfZf | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Interruptive Speaker) | H5ZWHom4mgdTkfAp5kxP9o | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Low Volume Speaker) | H2hKsYRLD6CPB6xYHie6px | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Standard Customer) | oL8EoeBMmwLKERjJRqzCWT | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |
| Test new prompt (Deepgram RBI Eval Agent, Super Fast Speaker) | 8WKhDBcmUzWn6UGYbJZM7s | 0% | 0/0 | canonical-v2, canonical-v3, popeyes, rbi-approved-scope, rbi-bakeoff, vendor-eval, voice |