You speak over the agent when you think you understand the next step. Interrupt naturally, then continue the conversation when corrected.
Test case names jump to that scenario's conversations below.
Failures first within each scenario. Simulation IDs open the transcript in Coval. The judge note is Coval's written reason for the Completion verdict.
Order five classic tenders by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose cole slaw and a Sprite.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
6zFxG2KaR9mAxYsCQ5XV7H | RBI Bakeoff 2026-07-23 | NO | Agent escalated to a human employee instead of directing customer forward without handoff. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B |
3aS6hwCpzw5osg89KYtiq3 | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent escalated to a human team member instead of directing the customer forward after taking the order. | 1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R |
noW5MKjKmKQeYCypkFxhm5 | RBI Bakeoff 2026-07-24 | NO | The agent did not direct the customer to pull forward/next window and escalated to a human assistant instead. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B |
X7VKH4QMcoNieUuKxS4Kyw | RBI Bakeoff | NO | Agent escalated to a human team member by stating an employee will assist the customer further. | 1x 5 Piece Chicken Tenders Dinner (Blackened Ranch, Bol |
eJXzf3ESoMmRFyc5Efi84a | Simulation Run 2026-07-20 22:35 (scratch) | NO | Agent escalated to a human team member instead of directing customer forward without handoff. | 1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R |
j3i4e7ca9sJsU9o7Rp9zmX | RBI Bakeoff 2026-07-24 | no transcript | — | |
4hHeXjuCaWsNDZf77PsDSF | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9kGUFpNEoRpc2UyRo3Uv8G | Simulation Run 2026-08-16 21:10 | no transcript | — | |
M5Ash3wbjzpGVpQEqFJmc4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
MH7GFCB7rMzUgNBaix4vAi | Simulation Run 2026-08-16 21:10 | no transcript | — | |
R6QDGf7dn9q6tvaLLpukZA | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UhmEVRLUWY6SRUKdxJkPqN | Simulation Run 2026-08-16 21:10 | no transcript | — | |
WEougGAT5QQ2LA7A2TfwzQ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Wbx4ki97SSD82VXZ3dkq2K | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YECDZ4mHBpMGUeyDcXAefD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
cE6rZUhC2wcNJo6BHoHhtn | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8PWVxREYhWPAKznXokJrNL | Simulation Run 2026-08-17 01:07 | UNKNOWN | Transcript lacks any order details or direction for the customer to pull forward or next window. | empty |
MHgu9FStf6e4ZN3zdaBA3i | Test new prompt (Aug) | no transcript | — | |
USxFHsFTGsRjWe7BSSePMr | Test new prompt (Aug) | no transcript | — | |
W6D9Mf9HzuVBrxxdhb9c6E | Test new prompt (Aug) | no transcript | — | |
WvsEH6nHRE8n8gzWBumGkF | Test new prompt (Aug) | no transcript | — | |
mqfSb6wLawuXLE98Ws8FXi | Test new prompt (Aug) | no transcript | — | |
eRoG2cLpS4EeR2JZr46uFt | RBI Bakeoff | YES | The agent fully captured the customer's order (5 classic tenders with two sauces) and directed them to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B |
Order one regular bacon and cheese sandwich by itself. Do not ask for a combo or a large. If the agent first offers to make it a regular combo, accept with fries and a Coke, then accept a subsequent l
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
aCohnPYtAV7LPA7jcK6ohH | RBI Bakeoff 2026-07-23 | NO | The agent modified the order to a 'Bacon & Cheese Chicken Sandwich Dinner' instead of the requested sandwich alone, indicating the order was not fully captured as request… | 1x Bacon & Cheese Chicken Sandwich Dinner |
LTvTf9oRhJWU5dKmaXkXnc | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent did not complete the order alone; they said, "An employee will assist you," indicating escalation. | 1x Bacon & Cheese Chicken Sandwich Dinner |
4qS8JktEibVCcXqFZrmbAd | RBI Bakeoff 2026-07-24 | NO | The agent repeatedly deferred to an employee to assist the customer, indicating escalation to a human member. | 1x Classic Bacon & Cheese Chicken Sandwich |
QVtSpUf2YKVsajqZPtkVBH | RBI Bakeoff | NO | The agent did not complete the order and escalated to a human team member for assistance. | 1x Bacon & Cheese Chicken Sandwich Dinner |
NY9Z5bszKzehraVhNd2Gct | RBI Bakeoff | NO | The agent escalated to a human employee instead of directing the customer to the next window. | 1x Bacon & Cheese Chicken Sandwich Dinner |
AHkambcpLZijRmP9AizGGW | RBI Bakeoff 2026-07-24 | no transcript | — | |
68eRJjLXMa82FhrNv94Zna | Simulation Run 2026-08-16 21:10 | no transcript | — | |
7dMxLRyYAQCrfoxpFGwweE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
B2ThWAvcWCErt25EajZiMJ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ByoKDDLKKA3Svj2MUgmJ6g | Simulation Run 2026-08-16 21:10 | no transcript | — | |
GAVbeXwgCwr7e5sK75kLkB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
V62Fc6KuWSYWvFZTy652Pa | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YHuyywFqPLsLX3DiEfdZ6o | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dx6vdDd2CHbArGwHgetfPF | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eWDhhY4VLvGBXhEr2NpZH3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hBXECNFGftr4482pAX764M | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UeWmEJgtHaYbozZXyiNXCn | Simulation Run 2026-08-17 01:07 | UNKNOWN | No order or conversation with the customer occurred to evaluate completion or direction. | empty |
7GCqP4qvxbAGdZXjdGuqJR | Test new prompt (Aug) | no transcript | — | |
FcK8SrNutfhvzTkVXSo2TA | Test new prompt (Aug) | no transcript | — | |
KahZg9nT2uKtzydqifFNWx | Test new prompt (Aug) | no transcript | — | |
UriEkQ7wLN9fcDqLLxAtJi | Test new prompt (Aug) | no transcript | — | |
nSXDpZ3hPWrBZeFGZKY7Rn | Test new prompt (Aug) | no transcript | — | |
n5xBdiFkvWHtUZ2EFh7LXF | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Bacon & Cheese Chicken Sandwich Dinner |
One bacon and cheese sandwich combo, large, with fries and strawberry lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
PsdhafuuUnoafRTfCdQi34 | RBI Bakeoff 2026-07-23 | NO | The agent did not confirm the full order by reading it back and repeatedly handed off to a human employee for assistance. | 1x Classic Bacon & Cheese Chicken Sandwich Large Combo |
CQzWGhLLUeX6aDbEc8khxc | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent escalated to a human for order confirmation and did not send customer forward directly. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
jX3JoEcM72iGmjnrYxuFfA | RBI Bakeoff 2026-07-24 | NO | Agent escalated to a human employee to assist the customer instead of directing them to pull forward. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
QgBHGChkg35xpCVpf4uBQw | RBI Bakeoff | NO | Agent escalated to a human team member instead of directing the customer to pull forward. | 1x Classic Bacon & Cheese Chicken Sandwich Large Combo |
ds6YSNs2FcU5FnTQ2gFBvt | RBI Bakeoff 2026-07-24 | no transcript | — | |
2W82A6eaBsTmCEFfwkwT8k | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8UdBNyzbYVMewCf3haeqTD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
C8P7kdEijkVroNuCMQTyhX | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RHg5BK6Tvt3HzmnAsyDHwc | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TgZDxYE8Zsy823L2B5yTNp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UNKydwDgujnu5hemcJ3nmH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UsZXKtZTaTZRaENq6pNetE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
VxxyF2aAfQSRjQCGNdjpd5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nAd4ZsGcWyNaXVEsSYFf6C | Simulation Run 2026-08-16 21:10 | no transcript | — | |
oT5gaCyG6Zo9hnewnVm8Au | Simulation Run 2026-08-16 21:10 | no transcript | — | |
S6FLXzzaune46ZPkCeu5z4 | Simulation Run 2026-08-17 01:07 | UNKNOWN | Transcript lacks order details or instructions about directing or escalating the customer. | empty |
SFeg4wp85fuKHbeztjcghU | Test new prompt (Aug) | no transcript | — | |
UzLysZpT7KUe5mbWYvgKse | Test new prompt (Aug) | no transcript | — | |
Wfdneiz4D3m5KfNvueEn6W | Test new prompt (Aug) | no transcript | — | |
ehJeN6AT2M83iMswYAQSBK | Test new prompt (Aug) | no transcript | — | |
nPLa8NonCQPZ9GyTjWyo9N | Test new prompt (Aug) | no transcript | — | |
AjP7UqLbmaQCJXZFmfNQtA | RBI Bakeoff | YES | Agent fully captured order, confirmed accuracy, and directed customer to pull forward with no escalation. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
bwUbu5Jr7x4z2gcWSZbj66 | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent took the full order, confirmed it with the customer, and directed them to pull forward without escalating. | 1x Classic Bacon & Cheese Chicken Sandwich Large Combo |
Can I have a Cajun Rice please?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
TvZr3txHnaQYuJR3NUxBrH | RBI Bakeoff 2026-07-23 | NO | Agent escalated to another employee to assist with order confirmation, not completing entire order themselves. | 1x Red Beans & Rice; 1x 3 Piece Chicken Tenders Regular |
n5jSqxpUMVVfe5L8LHRYXE | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent did not fully complete the order independently; they escalated to a human for assistance. | 1x $5 Favorites - 2 Sides (Red Beans and Rice, Cajun Fr |
hAVa6KL2wVSRrcWaZ4vLQN | RBI Bakeoff 2026-07-24 | NO | Agent said 'An employee will assist you,' indicating handoff to a human team member before directing customer forward. | 1x Red Beans & Rice |
SWsn3okuGN9Zki7j2MKBEa | RBI Bakeoff 2026-07-24 | no transcript | — | |
2myYYcFmWdyZzvVewPeFn9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
47VzkhMGQYWSLm8YYrFwV8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
AkMykz7Gw9PPW5JD6KJGTd | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HNoqEKZbyGqUJpjcTjasEt | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HkwgL7749Ts3huGohWNE4v | Simulation Run 2026-08-16 21:10 | no transcript | — | |
L9ritLEbXSscj65rWTLcZQ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PDMBBT8Y8QoTH4aY6JPgPL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QvoVsGfGwnxKsmwYiEwDbP | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript ends after order initiation; no indication of order completion or directing customer forward. | empty |
RY6LajXiLoqVTdTqQZLoCG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UyJ5GJqnRvSov8HN5re7S8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9JuSz3DXqWBpqKMHP5zhiF | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the full order being taken or the customer being directed to pull forward or next window. | empty |
AhPHpoz7KFudfSgWNTx5Wb | Test new prompt (Aug) | no transcript | — | |
C5pjcbdFEr7nnXZ9k3m8Xd | Test new prompt (Aug) | no transcript | — | |
Ck8UHYComqSqhpAvEVY44f | Test new prompt (Aug) | no transcript | — | |
LfLXbMngezT4cBDsYXtoZi | Test new prompt (Aug) | no transcript | — | |
drRGz8KpC6u97EhqWtub7T | Test new prompt (Aug) | no transcript | — | |
nKdQhhJEi65DgHrFK6suwY | RBI Bakeoff | YES | The agent fully captured the order and directed customer to pull forward without escalation. | 1x Red Beans & Rice |
Urr2d2VPELQ2ie6KVgmhDA | RBI Bakeoff | YES | Agent fully captured the regular red beans and rice order and directed customer to pull forward without escalation. | 1x Red Beans & Rice |
7RTT2ky8bCAjiZXqojMcpg | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent took the full order (regular Red Beans & Rice and a biscuit) and directed the customer to pull forward without escalation. | 1x Red Beans & Rice; 1x Biscuit |
I'd like the 5-piece tenders combo, blackened and spicy, with cole slaw and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
MfqJpvw7iDyK2jhC3FQfkD | RBI Bakeoff | NO | The agent escalated to a human team member instead of directing the customer forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
J9EZnexVDGiuAXxxVx3dWD | RBI Bakeoff | NO | The agent escalated to a human team member for order verification instead of sending the customer forward. | 1x 5 Piece Chicken Tenders Large Combo (Blackened, Spic |
SzZ7hZc7aYEmMaoXRgGqgj | Simulation Run 2026-08-17 01:07 | NO | The agent never directed the customer to pull forward or the next window and did not confirm the complete order before ending. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
cCB7ePAnTfMSvkAJ8kVdVv | RBI Bakeoff 2026-07-24 | no transcript | — | |
4BNh6G5jAduGBhtx7bYQjd | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5SFPhF23jqHxYFMw2KdbZE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
BTefWBE7kAdud4zwDDUS6Z | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HohrzqEbzN5GWR9mrSvGsm | Simulation Run 2026-08-16 21:10 | no transcript | — | |
LN4aEhyi6uocgyS8F8FTbm | Simulation Run 2026-08-16 21:10 | no transcript | — | |
WrNLm5vDNYebiMnzeV4eSB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
XhKXb2cPMQQzdFJy8HDYBH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YS6dBsmD4UHQsXwPVzA7xN | Simulation Run 2026-08-16 21:10 | no transcript | — | |
gS2x6LF8xoxoJjxrZDxEsu | Simulation Run 2026-08-16 21:10 | no transcript | — | |
jFHVQa7U4XqT6Hcpuuju2j | Simulation Run 2026-08-16 21:10 | no transcript | — | |
3FZn2XmXJruZrQwuAxxzoW | Test new prompt (Aug) | no transcript | — | |
BA72CLNHEby5ebZdYb8nVu | Test new prompt (Aug) | no transcript | — | |
Rc43vZPBVU4pTCAZ8tEVuQ | Test new prompt (Aug) | no transcript | — | |
dAyHQjtVpfwGFfm68kPJnE | Test new prompt (Aug) | no transcript | — | |
dKPgxJQVqEZv57Qzvvg4R6 | Test new prompt (Aug) | no transcript | — | |
6jAhKvHDjiBRcFDyydA4xo | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
kWfqEadvarSoTBAhe7wmpa | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order and directed the customer to pull forward without escalating or handing off. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
8L34WhyJEzsAecBvxnPL2B | RBI Bakeoff 2026-07-24 | YES | Agent fully captured the order and directed the customer to pull forward with no escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
MwiUNYKPcsdNaxgybL6fdW | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the customer's order and directed them to pull forward to the window without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
Order three pieces of spicy signature chicken by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose mashed potatoes and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
YSKSLEwi59VPstLzfcALFy | RBI Bakeoff 2026-07-24 | NO | The agent told the customer to pull forward before the order was finalized and modified to a combo, not completing the full order then directing forward. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
fwBtmD9RM2CPMG7UwzgPLR | RBI Bakeoff | NO | Agent could not read back the order and directed customer to screen or employee assistance, not fully completing order themselves. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
Qy5yL5iUwh4yRWp4MhqXaw | Simulation Run 2026-08-16 21:10 | NO | The agent escalated to a human by saying an employee will assist the customer. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
mXwWk7nZ3JTgpUe8icau3S | Simulation Run 2026-08-16 21:10 | NO | Agent asked if order was correct but refused to read it back and suggested getting an employee if incorrect, indicating potential escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
GMZ4knYQruUS9mi2rxSyo2 | Simulation Run 2026-08-17 01:07 | NO | The transcript ends without the agent directing the customer to pull forward or the next window. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
Q5rR2yNoGkaSzwZ7hbkWMV | RBI Bakeoff 2026-07-24 | no transcript | — | |
2KvxbLzyjDHWiFVhHe27uh | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Mp6a9fDoETHeegV9GQJamZ | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript lacks confirmation of order completion and direction to pull forward/next window. | 1x 3 Piece Signature Chicken (Spicy) |
3M592Q4J3uFQ7wLwTYREAG | Test new prompt (Aug) | no transcript | — | |
CLmJ2yux58cwDeZenVZXg3 | Test new prompt (Aug) | no transcript | — | |
SmM7n7nJ8kaUWKkAev9rJ3 | Test new prompt (Aug) | no transcript | — | |
WDgSGZEBjghGwBdiv2WeeN | Test new prompt (Aug) | no transcript | — | |
jKQ9bPJy6RCQ23fttTj5Ue | Test new prompt (Aug) | no transcript | — | |
ZuNLLGLGduMDuV4sXaHF4j | RBI Bakeoff 2026-07-23 | YES | The agent captured the full order including changes and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
ccFm4PuyXqRjYHCFN34soQ | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
noRwXzZUbPxtGKSbh9Q5Lu | RBI Bakeoff | YES | The agent fully captured the order, confirmed correctness, and directed the customer to pull forward, no escalation occurred. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
B5wNyF9LUktZLBLYmVeynk | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent captured the full order and instructed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
EVdcaVvLybZ5azX58FVXnt | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order and told the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
eDq2LW3mLHqEfPdzFcJHNW | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
gBivbrHmCZYxpivfkyN6n2 | Simulation Run 2026-08-16 21:10 | YES | Agent fully captured the order and instructed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
jKEcPU39MUzevnapQM576D | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
ko3PF9QLNVWP39TDLViTh3 | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
o6guRxWP5EBUMzuzjinLG2 | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
I'll do the 12-piece family meal with all dark meat, spicy, Cajun fries and cole slaw, and two Sprites.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
Y6FLQNZdHKuEZ775jEYMC3 | RBI Bakeoff | NO | The order was not fully captured; the quantity of the 12-piece meal was only recorded as 1 instead of 2 as ordered. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
nh4sNhAXFfqoBdFVbsuvCs | RBI Bakeoff | NO | Agent did not fully capture the two Sprites (only one shown) and did not explicitly direct customer to next window without escalation. | 2x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
RwTZa6DqEJL24zR6LMuqm9 | RBI Bakeoff 2026-07-24 | no transcript | — | |
2iYStp7AFActRtAv23qH9u | Simulation Run 2026-08-16 21:10 | no transcript | — | |
4FnkWTBkMmPdVijNTBUibU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9kwzxj6o26nNWfdEszXBNR | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9pxpgxZYLjKruqgYbWA59L | Simulation Run 2026-08-16 21:10 | no transcript | — | |
DJufB4ogGYUQKJaKDvtdJD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NZW5Z2mvVL886kmRjg7eeB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fE7tC6GGXJ7hucwE4iwxAF | Simulation Run 2026-08-16 21:10 | no transcript | — | |
gBhzPgEiLRJnnacP6srsDV | Simulation Run 2026-08-16 21:10 | no transcript | — | |
j5hJ5KcBra8gJvSniPgEU8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
oFQBKHLS5jrfLKACtv4pH6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
kvq8yJf9t6ujyU3gCvjTkG | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript contains only customer order and agent greeting without confirmation or direction to pull forward. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
EYFhzBGcLk87av9NULCqsy | Test new prompt (Aug) | no transcript | — | |
N3HS6zCGfny5dF2HK5sb2v | Test new prompt (Aug) | no transcript | — | |
PzNWZDBSV5y98d4EGVtLio | Test new prompt (Aug) | no transcript | — | |
WSSB22gqJpRfnnxvPCM9QP | Test new prompt (Aug) | no transcript | — | |
ac3ziMYzMDSpsMrCPhbXQL | Test new prompt (Aug) | no transcript | — | |
9Ts6PRZwcxxXoVVgKyjyPf | RBI Bakeoff 2026-07-23 | YES | Agent confirmed the full order and directed customer to pull forward with no escalation. | 2x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
hmbcvYDLLJfe6ueXKcDZH7 | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
fwqjV5a47NaTFVuKoV2eMt | RBI Bakeoff 2026-07-24 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 2x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
fDY9Rd93aTs74y8yMByDeE | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent confirmed the full order and directed the customer to pull forward without escalating. | 2x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
Can I get the ghost pepper sandwich combo, swap the fries for red beans & rice, and a Dr Pepper?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
8SAgx7VazhuawYPjQJYZNg | Simulation Run 2026-07-20 22:35 (scratch) | NO | The agent confirmed the order but did not explicitly direct the customer to pull forward or proceed to the next window. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
e5HBPfNBPfojfAt6gPnfSV | Simulation Run 2026-08-17 01:07 | NO | Agent never acknowledged or confirmed the order, nor directed the customer forward or completed the order. | empty |
ZmH7VpEomtrfPzUYQgtbAK | RBI Bakeoff 2026-07-24 | no transcript | — | |
3vSMjayrzHECweYZanQkFT | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HJtaVXHuRXvy4UtERwGmsP | Simulation Run 2026-08-16 21:10 | no transcript | — | |
KWsiZyKyijJ82SkmseMZnb | Simulation Run 2026-08-16 21:10 | no transcript | — | |
LCZwWHq8U29nJKD44XCtsW | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QYqTUQsTJkd7gpP5sGQcUe | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Rd263woAnwEQvHny7zVfjk | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dU9dMay6aoTPyjWvnUVwrQ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
iTCmqMZd2RuSe5PMq7bBJ2 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
kh8dBhS44wUEmPqfLvGf2T | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nibqHR84szsjxyt8MNZq2T | Simulation Run 2026-08-16 21:10 | no transcript | — | |
3jrwX2aD2isyrzn4Kqi2D5 | Test new prompt (Aug) | no transcript | — | |
7tf4wqasCpiagMBqTpqRHs | Test new prompt (Aug) | no transcript | — | |
8QYSSVaGmNLE3LFB6SCCeq | Test new prompt (Aug) | no transcript | — | |
JzRWAiFTsMr9QcbMmUwsDT | Test new prompt (Aug) | no transcript | — | |
RConEKo5LE6j37hj4rs9rx | Test new prompt (Aug) | no transcript | — | |
ZB9k8nK93yv4UqMfgDAaP4 | RBI Bakeoff 2026-07-23 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
ioUE4G7DLiyJ5X5QmoJSDN | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent took the full order and directed the customer to pull forward without escalating to a human. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
kuv4HpsXjbM3RcohPSg8Rz | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order, confirmed it with the customer, and directed them to pull forward with no escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
JJPKnZQnoUytBSJo3voWpw | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
5HicmEvwxdbF95hHsz3p85 | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
I'd like the 3-piece signature chicken combo, spicy. I don't want fries. What other sides do you have?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
ANdwgLBQEtDV5bohdghRds | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent escalated to a human team member instead of completing the order and directing the customer forward. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Red |
Da9v3sM5pRC78xLkgiE5tg | RBI Bakeoff 2026-07-24 | no transcript | — | |
74ksWvSnGitb9MTh3ZCNNx | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8fRSQgMADzTR9HarFBuU5K | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Api5NhcVEZcKb4QAsvKPvm | Simulation Run 2026-08-16 21:10 | no transcript | — | |
BeoNAZ569GGDoGS9ZkenKE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Eah62oFZQoukDFbENGLuwo | Simulation Run 2026-08-16 21:10 | no transcript | — | |
LwdVvox9tuWfpR4qdiu7Ba | Simulation Run 2026-08-16 21:10 | no transcript | — | |
MG2Zvw4JrSNRcrRcM9V7Jx | Simulation Run 2026-08-16 21:10 | no transcript | — | |
aJPZJvv5kyyuPSsM22ETSK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
f3FJPRpFxTZy3ehB5MGBdB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
itCbsNyp7Gv69nDZ5G5bvc | Simulation Run 2026-08-16 21:10 | no transcript | — | |
MMKCpBtf2sH2FSiVVPXnCH | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript only shows the customer ordering and asking about sides, no agent response or order completion is present. | empty |
BcKiYGrZYhKgKw7bVqKGgn | Test new prompt (Aug) | no transcript | — | |
P4677XhXSSRwzVpJnMDW9N | Test new prompt (Aug) | no transcript | — | |
UhJYchD2WFxjSnzqwXkbqa | Test new prompt (Aug) | no transcript | — | |
aWRGbzK3YC489AmFmxHSXH | Test new prompt (Aug) | no transcript | — | |
bK22D34D3TufJZUz2LXkQa | Test new prompt (Aug) | no transcript | — | |
5hkfXB8VgCCitdY82WX3iL | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
eUMfaY2gDZg3FLeV6pGwqp | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order and directed the customer to pull forward without escalating to a human. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Red |
Zg8e9DTRQ7CxBizd84futu | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
AyvUqoaHpoQe4CxdxrrmnP | RBI Bakeoff | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Home |
YgRHHVLzBmwr7s7bDyUn3T | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the order and told the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
Order one classic chicken sandwich by itself. Do not ask for a combo. If the agent offers to make it a combo, accept and choose Cajun fries and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
9NMhouX5wgaxSWwMXyctgD | RBI Bakeoff 2026-07-24 | NO | Agent could not read back the order and deferred to employee assistance, indicating escalation. | 1x Classic Chicken Sandwich |
UxELHK3tVcvovqDzVKx5jn | RBI Bakeoff 2026-07-24 | no transcript | — | |
3ZQiTaEvmmAXYaUoRuyWZU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5uHqtSJy9gtJNLg3AvrjsW | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Soynd8aaqXc6yigcvAfU4K | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Wiv8FyuSLt3KQUagygn4VE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
baaih9ouxnHWaepXKauuE8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hbVVR4hiVx6oiSoUhNoHHa | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hzFnvtUjQFimAyiSErSc7n | Simulation Run 2026-08-16 21:10 | no transcript | — | |
m7gBTvEnXWdECFaCTUfkoE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
mXSiLEbiguRXhfJCyxSCsx | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nnkvUdxWEWMNmSPZX3kgeg | Simulation Run 2026-08-16 21:10 | no transcript | — | |
jpuwr6weu94nS9M3yhdRmK | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript ends before the agent completes the order or directs the customer to pull forward or the next window. | empty |
9FY6KUi7EkVxZGGaH6ujBG | Test new prompt (Aug) | no transcript | — | |
PYXquietqi7EAGCJY4GicZ | Test new prompt (Aug) | no transcript | — | |
RkAUYgUf2ZKDSKnXk3hgV8 | Test new prompt (Aug) | no transcript | — | |
WeUxTwoFg8hu7K566q2mmG | Test new prompt (Aug) | no transcript | — | |
XbLNYi65DSLJJyMLkcyCAA | Test new prompt (Aug) | no transcript | — | |
BS5Mimdh4ViNRmvr8V39Uj | RBI Bakeoff 2026-07-23 | YES | Agent fully captured the one sandwich order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
5FCAvh3keXWsb6Y3u7R4Qz | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order and instructed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
9WnCQUmYrsqSatagGvtALw | RBI Bakeoff | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
6GHz828bkNWesyiZY7qFDh | RBI Bakeoff | YES | The agent fully captured the single sandwich order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
ZxKFdbTipQGtHVRUZka8WV | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent captured the full order (classic chicken sandwich) and directed the customer to pull forward without escalating. | 1x Classic Chicken Sandwich |
Give me the chicken wrap bundle with a mango lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
ew7FM5Am9dPurpytCCqVA4 | Simulation Run 2026-07-20 22:35 (scratch) | NO | Agent escalated to a human by saying an employee will assist and did not complete order delivery. | 1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C |
ZBcBq6TPQPuk8SoUmAYQGk | RBI Bakeoff 2026-07-24 | no transcript | — | |
2QVaSP7WSgiATogsL7XiMD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5wFSVQ4T2botSojVVFcvSH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
6sWyFnqqVZ5xAhfy7cLS2P | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9RT62aMhkgXWxQFd2vYHZa | Simulation Run 2026-08-16 21:10 | no transcript | — | |
JBcUzYqaKAKoVi5U3nndjU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QkswECLzhhJdKMDjCpuPZo | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YFjxGCX25UrEJbS9GCLnhe | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZLbdQRk4kika8G62YjHY2A | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dEHt3LQbKvrKrBdzBi5RUt | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dVnkJveUSJkM6yaxreJ6RE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
K53sb5drhcweFQnRCRCRXd | Simulation Run 2026-08-17 01:07 | UNKNOWN | Transcript does not show order completion or directing the customer to pull forward or next window. | empty |
5YcsXztquJ8jv7qTeTWvFh | Test new prompt (Aug) | no transcript | — | |
CKyx4gHRwRo4EoAMgiYmuk | Test new prompt (Aug) | no transcript | — | |
PBoQvvmzZ48LNKnTsJRQpG | Test new prompt (Aug) | no transcript | — | |
WbWm636oQVLDiFprEXUVwb | Test new prompt (Aug) | no transcript | — | |
hzwhQnqmhRw9gKCQkAPJHL | Test new prompt (Aug) | no transcript | — | |
cVLNaQNuGMMaVb3oPyjo7D | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C |
as3oUbZVDYMUiGRnhiGPe8 | RBI Bakeoff 2026-07-24 Pass 5 | YES | Agent fully captured order details and directed customer to pull forward without escalation. | 1x Wraps Bundle (Frozen Mango Cane Sweeeet Lemonade, Cl |
6CmXcqRtTkXTSUSozgzLGR | RBI Bakeoff 2026-07-24 | YES | Agent fully captured order and directed customer to pull forward without escalating. | 1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C |
ZBfbDmkhy3YoFBfA6BdaBS | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C |
2vqfbjyGncQCU3JtcZ49fS | RBI Bakeoff | YES | Agent fully captured the order details and directed customer to pull forward without escalating. | empty |
I'll take the 3-piece chicken combo, spicy, add a pie - make it the strawberry - and a premium strawberry lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
9p9yuQFfk27X3oGZiVPeWu | RBI Bakeoff | NO | The agent failed to read the order back and suggested a human could assist, indicating a handoff or escalation potential. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Cajun |
GAGAAAHXRMhDSDDjPa5xph | RBI Bakeoff 2026-07-24 | no transcript | — | |
BYhFKup4vXUqzN8abu95S2 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
E8xNX9aK4aH3xdejzrEMoM | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript is incomplete; no confirmation of full order capture or direction to pull forward is observed. | empty |
Emu4NXyYiorH9nrTua7iFT | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript ends before the agent confirms the full order or directs the customer to pull forward. | empty |
ErKCnhPPLHHPmwrpredK4F | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HTygRHP6qg4u5S2FdT3r4Z | Simulation Run 2026-08-16 21:10 | no transcript | — | |
KbxWNFHimn4TzaBNuewcWw | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QYURRMeLVjy2VV4EZtKdaD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Qxv4npfxj5w6TSCEVK5q9Q | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Szj5JopiV92VTaciHmFW4e | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZyHNMKwU6r4M5XjJgJKMPP | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PPCqE7tjqhNicmdLevyQB4 | Simulation Run 2026-08-17 01:07 | UNKNOWN | Transcript only shows customer order, no agent response or direction is provided. | empty |
GEqmSNnb9nDxRX9nmVTKuN | Test new prompt (Aug) | no transcript | — | |
RAsmr5H23SgE59PKRdsEJF | Test new prompt (Aug) | no transcript | — | |
cPx9f3Rj39z39zX56PZJoi | Test new prompt (Aug) | no transcript | — | |
goTicsyyKzhVuJ4PgwkY7B | Test new prompt (Aug) | no transcript | — | |
hhRDgwrjZT6jNiUTyopFKe | Test new prompt (Aug) | no transcript | — | |
hsG4hLwXJ8w9gzc3ARBZ7D | RBI Bakeoff 2026-07-23 | YES | Agent captured full order details and instructed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
F2m6kPD7KuxMxayJEnCGuH | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil |
2Xn4vC53sckpNZhB7tE7ET | RBI Bakeoff 2026-07-24 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Caju |
VDgE3C6t7sAyCP9brxFCWF | RBI Bakeoff | YES | The agent took the full order, confirmed accuracy with the customer, and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Frozen |
HMjqVHNZkNBfGMtzy9gNAB | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Froz |
Do you have nuggets?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
WeSWo5dMrQcH93FqNqZnHo | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent escalated to a human by saying 'An employee will assist you' instead of directing to the next window. | 1x 3 Piece Chicken Tenders Regular Combo (Cajun Fries, |
mt3P4WjvLqzRcGrjxYC2w6 | RBI Bakeoff 2026-07-24 | no transcript | — | |
4ik6qKLhKhEpMnEvQLiE7U | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5Ay5x7jA4934Yp5RYXNYsp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5b8XVjfc4cMt73HL5Lq84s | Simulation Run 2026-08-16 21:10 | no transcript | — | |
6FtPfCAtx6jdhcGts9FpMm | Simulation Run 2026-08-16 21:10 | no transcript | — | |
BZFaFbadsGoh8AmtuC7iae | Simulation Run 2026-08-16 21:10 | no transcript | — | |
C7NLysSFfjtKi92uwsHD8h | Simulation Run 2026-08-16 21:10 | no transcript | — | |
CPn6z9PcZvMZH2snn9YG2Y | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RURNpiiZjyyJPUr6P29Bgq | Simulation Run 2026-08-16 21:10 | no transcript | — | |
b4Gz4aAX6dtZ4J4KMMiQ2V | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nPo8o5yirJ2Y5xgw9Tcssh | Simulation Run 2026-08-16 21:10 | no transcript | — | |
KHgwH7JU4LQpPuDxAQbrUD | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript contains only greetings and clarifications, no order taking or directing the customer forward. | empty |
3cMXpLsyYwHKjZeErvN3cd | Test new prompt (Aug) | no transcript | — | |
NFDHxUpvJMDutQJkTuoPgJ | Test new prompt (Aug) | no transcript | — | |
P3sMKYz8gtq8qpMVFmRN4x | Test new prompt (Aug) | no transcript | — | |
Tcm4CNr2TUbDzqNRawSU7u | Test new prompt (Aug) | no transcript | — | |
ZtEZvX6pyESUnaU2HjCEoq | Test new prompt (Aug) | no transcript | — | |
NEQvg8yveee934oWEGDv4o | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order including details and directed the customer to pull forward without escalation. | 1x 3 Piece Chicken Tenders Regular Combo (Homestyle Mac |
kynXBgEUamG9D7zbJQRBqX | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the customer's order and directed them to pull forward without escalating. | 1x Surf & Turf Large Combo (Bayou Buffalo, Blackened, G |
KaqEuvnoZjRj78A8iQx69K | RBI Bakeoff | YES | The agent captured the full order, confirmed it, and directed the customer to pull forward without escalation. | 1x 3 Piece Chicken Tenders Large Combo (Classic, Cajun |
jsvxzy3xSBM9g5fZgXn8c5 | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Blackened, Blackened Ranch, |
9vKPjQXWN3eNnQ8TU6G4NV | Simulation Run 2026-07-20 22:35 (scratch) | YES | Agent took full order with modifications and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
Start by ordering the 3-piece signature chicken combo, spicy, with mashed potatoes and a Coke. Before the order is finalized, change it to the 4-piece combo.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
PjGxxMbBYmdGScHVeyVNYx | RBI Bakeoff 2026-07-24 | no transcript | — | |
8yQPzFBUomPcEsfWTDNMaQ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
GNqDbaKQXNvmkS77PBKgY7 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
R7E2W4yLB3EVJM5MMM7z2w | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RpVkEUB2vXFm4yZGdaLJTL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TVBvDqMhZMyBeCdNGrQMs6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UGG9zZpqZLG7em58rrBJFK | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript ends before the order is fully captured or the customer is directed to pull forward. | empty |
WPhMzjAokmvpiuhRLnYkpE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
h2WNtyjmgsXdZTXQ8RXC3L | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript does not show the agent completing the order or directing the customer to pull forward or the next window. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
ijwEV5xeBpeiYwKyCsFia6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
mMx2gtNmqK2kX4jS2MMx9i | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript ends before the agent confirms the full order or directs the customer to pull forward, with no escalation evident. | empty |
9mdXP9ruzER2AbNK8XasH4 | Test new prompt (Aug) | no transcript | — | |
ELuk4CdNsBwD4hhszFeRHw | Test new prompt (Aug) | no transcript | — | |
auDVfQ8vXW8y4ySGgbrqiP | Test new prompt (Aug) | no transcript | — | |
cirKeCqbKZWcJtUFwgJjxb | Test new prompt (Aug) | no transcript | — | |
iFH8SMj7MALQnGrqc2Ez7n | Test new prompt (Aug) | no transcript | — | |
QBHWJe2cwRW2PZDn2Rtq4d | RBI Bakeoff 2026-07-23 | YES | The agent captured the full order, confirmed it, and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
GncfKVm8E6NzH35KPAjcbV | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent took the full order and directed the customer to pull forward without escalating. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
mAkZm8huhj4mPUMVckmzhg | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order and directed the customer to pull forward without escalating or handing off. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
G47UjTEyKDE72ttoaCJ3BF | RBI Bakeoff | YES | Agent captured the full order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
SSecVy9kRAaZNhK8cyjEJh | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
2nWcE6uWQaqZifojz2z9YA | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
P5Rc22Kj598Nzn6gLKKr4i | Simulation Run 2026-08-17 01:07 | YES | The agent captured the full order and directed the customer to pull forward without escalating to a human. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
One Ghost Pepper Sandwich, no combo.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
2RTCiPMXZQuVH8HzwpMkEU | RBI Bakeoff 2026-07-24 | no transcript | — | |
39eZ5efG3ZEPwjCrHgxvcE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
4aR8iWmpYs7dbFJLKVWLVS | Simulation Run 2026-08-16 21:10 | no transcript | — | |
GRejQUwM4aPyxhzDJPL8bP | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TK75S6ako5svG6tFgq9TcL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TWMHPmVE9Rzm4o9ezubnAY | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UYoYeYqnbLZ2cjZSJXgwSi | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Zqcg6uFSGpBQv6mwj3SJDE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
bAJPe7ZaG5EHyq5zqXFo8E | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eKhiWqHReD4UuY2JPeJkQ4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
n2Xtq9oKhKoaD9eiVPjXfg | Simulation Run 2026-08-16 21:10 | no transcript | — | |
97Lyp256ZL6kyHGasCxycQ | Test new prompt (Aug) | no transcript | — | |
JZqNXTj9oMBzF3vCjAujhk | Test new prompt (Aug) | no transcript | — | |
T4kupYccnuxe8bkK3aGZaJ | Test new prompt (Aug) | no transcript | — | |
gW4KaYCsMuKHrnoZcA7WJZ | Test new prompt (Aug) | no transcript | — | |
kpcUC23bAwCQ6x72zydJY7 | Test new prompt (Aug) | no transcript | — | |
KWA3nfdQ8GzdaitRgfQzkE | RBI Bakeoff 2026-07-23 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
UQgcnUb5bow4BnKrGEYSLY | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
mXmcPAcazezLi4deJrc6Ye | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
G2Na3BdzTUYsMXjtYiLe8f | RBI Bakeoff | YES | The agent fully captured the order (spicy sandwich, no combo) and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
kLHVwzsrgkFi9tk9LPfPRE | RBI Bakeoff | YES | The agent took the full order and instructed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich |
5G2WQ9WWMkZsENm4BGJ7CL | Simulation Run 2026-07-20 22:35 (scratch) | YES | The agent fully captured the spicy chicken sandwich order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
ZiZPQJbpznCYEzsFeFMhCe | Simulation Run 2026-08-17 01:07 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich |
I'll do the chicken sandwich by itself and a side of fries.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
jqAgGqCDgMyoJEFAkxgPBy | RBI Bakeoff 2026-07-24 | no transcript | — | |
2r8VspoHHu9DBFCYZ6W5Nf | Simulation Run 2026-08-16 21:10 | no transcript | — | |
6eCFmhUjEUeFBZsFkh4rZf | Simulation Run 2026-08-16 21:10 | no transcript | — | |
A8rRBwKGi8Cm9tsQBbZYFc | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Ng84uxRSCbhMmpnXw58Cjb | Simulation Run 2026-08-16 21:10 | no transcript | — | |
W82MEutDjm4Pr3NLHpgUeV | Simulation Run 2026-08-16 21:10 | no transcript | — | |
WQJAgMkiXxTRMtzKo9ZFyR | Simulation Run 2026-08-16 21:10 | no transcript | — | |
XhwzXvPJmsdDeHzcYCV9cB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZamNajohPFxbSU8HARUjFN | Simulation Run 2026-08-16 21:10 | no transcript | — | |
aDYQxCpv7pYJ2N9w2Z6AXn | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eTsTWEGodP6fr5eTbD9K39 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
3kGM9xaDApKJiNknEm8Vh3 | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the agent directing the customer to pull forward or the next window after taking the full order. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
Pg79JKbmtwtxW9ydqiAkaN | Test new prompt (Aug) | no transcript | — | |
VQW7uQR8UpMQEsq7XJx3oD | Test new prompt (Aug) | no transcript | — | |
Vq6NHpwaXnWjPz8of5sbsF | Test new prompt (Aug) | no transcript | — | |
b6D5GwBRZAnok8u74hcB8y | Test new prompt (Aug) | no transcript | — | |
f8B4VjwHWH2UZNKJAiVvKK | Test new prompt (Aug) | no transcript | — | |
2YtJpGwnSGD5qa6wQsccji | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order details and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
dahK6BiimqQoHGCSbXcyCQ | RBI Bakeoff 2026-07-24 Pass 5 | YES | Agent confirmed full order (classic chicken sandwich and regular fries) and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
ZTXxUD8VSBgXit5fadMqau | RBI Bakeoff 2026-07-24 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
URHRU8wpYcVqhRhv6QybDK | RBI Bakeoff | YES | Agent took full order with classic chicken sandwich and fries, confirmed order, and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
Y4d93L7yPyXD3P8umB74vM | RBI Bakeoff | YES | The agent fully captured the order of a spicy chicken sandwich and fries and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich; 1x Cajun Fries |
k4HntLbJmcAP2VZcNVcdAL | Simulation Run 2026-07-20 22:35 (scratch) | YES | Agent took full order including fries and ranch, confirmed with customer, and directed to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries; 1x Butterm |
Can I get the Ghost Pepper Sandwich combo? Actually, change that to the regular chicken sandwich. And make it spicy.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
Ve7a25DCxSabpp9Uby4yBK | RBI Bakeoff 2026-07-24 | no transcript | — | |
6i6cgMn3PEWy6DoxSueaKH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
CAQbHbWQXRATKe6ArdkB46 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Durff7PCP78VZAdeQbwfmh | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EmEWYzyiWTa5xhfYwdqLGq | Simulation Run 2026-08-16 21:10 | no transcript | — | |
JjozxJVPhGEG8EU7JP3iYH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NKigf3DEc8yi7xtBhgUyET | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RVVnQhTMo2jXVun9TkpW6y | Simulation Run 2026-08-16 21:10 | no transcript | — | |
WjfnVtZtQTo6mKDoWjLbtd | Simulation Run 2026-08-16 21:10 | no transcript | — | |
chn5ndvk5nuxoRmsPFSYDh | Simulation Run 2026-08-16 21:10 | no transcript | — | |
jt5oEYvyd3FVVKm6vBbse4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hnN4kaCddKqcxscVQvpfdP | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript ends before the agent confirms or directs the customer, no full order completion or direction is evident. | empty |
EhbeseJnAfagQi7ay2Bdq6 | Test new prompt (Aug) | no transcript | — | |
T2zgxbzZHWT4uPM8H7L4rG | Test new prompt (Aug) | no transcript | — | |
ecn2ghy7XPmZ9p4EUjQCi2 | Test new prompt (Aug) | no transcript | — | |
em9rKbEJkFhF3ADR8YhKHV | Test new prompt (Aug) | no transcript | — | |
fuidiaS4yHwZYMs9sCGKEY | Test new prompt (Aug) | no transcript | — | |
3a6x3MdrtY2982BhisL2Hx | RBI Bakeoff 2026-07-23 | YES | Agent fully captured the order, confirmed it with the customer, and asked them to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Mas |
SUT5YEv5xMbNvwVAZ5PSEN | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Mas |
bZ8ttPctm4L9YXNxt56gTC | RBI Bakeoff 2026-07-24 | YES | Agent captured full order details and directed customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Large Combo (Homestyle Mac an |
QVRtB4gZCXwDqwKNBzcqg2 | RBI Bakeoff | YES | The agent took the full order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, F |
TaJL3RPki59dgQgwK9Rmgb | RBI Bakeoff | YES | The agent fully captured the order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Mas |
AKjw6TtvH9rLVA78M63rEc | Simulation Run 2026-07-20 22:35 (scratch) | YES | Agent took full order, confirmed details, and directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Frozen Mango Can |