Speak naturally and conversationally like a real drive-thru customer. Stay in character and only order/respond as a customer would. You speak English with a Hispanic/Latino (Mexican-American) accent; English may be your second language.
Test case names jump to that scenario's conversations below.
Failures first within each scenario. Simulation IDs open the transcript in Coval. The judge note is Coval's written reason for the Completion verdict.
Order five classic tenders by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose cole slaw and a Sprite.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
TzPAFksD8Yf4BHjpWzkhCd | RBI Bakeoff 2026-07-23 | NO | The agent escalated to an employee to verify the order instead of completing it themselves. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
9AmPAFFF77Tgsj9AUHKiwH | RBI Bakeoff 2026-07-24 | NO | Agent confirmed the order but the items shown included a combo dinner, not just tenders alone. | 1x 5 Piece Chicken Tenders Dinner (Blackened Ranch, Swe |
ZBLBgDuuU9cgiX38DcLGsJ | RBI Bakeoff | NO | Agent escalated to a human by saying an employee will assist the customer. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
YjJP6rfLhzTCmqkxwh7jui | RBI Bakeoff | NO | The agent said 'Employee will assist you,' indicating escalation to a human team member. | 1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R |
JvEpsQaxN8XFxnVbcNfThb | Simulation Run 2026-07-20 23:03 (scratch) | NO | The agent escalated to a human multiple times instead of directing the customer to the next window. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
BCfBYeqYsbMyJg3pWsrJ89 | Simulation Run 2026-08-17 01:07 | NO | Agent did not confirm the order or direct customer forward; only repeated welcome message, no order completion. | empty |
d2XYbEyVqzBELpvPhihcP3 | Staging (larger sample size) | NO | The agent did not fully read back the order and directed the customer to an employee at the window. | empty |
fKkgqWpSVvxSW7Ce5GdbcN | Staging (larger sample size) | NO | Agent failed to confirm order with customer and escalated to human team member for help. | empty |
RVgEWScefyQcC4AH8VHTKE | Test new prompt (Aug) | NO | The agent did not process the order or direct the customer forward; no order completion or transfer occurred. | empty |
3wRDNMUSPKmLPTSvYF84WC | Simulation Run 2026-08-16 21:10 | no transcript | — | |
DK2KTakE3YqaxCbyuQuxgw | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NMBvSYtrCaF3Z5b86wtiec | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PhZrbZUMv9ZwZrAjnazGN7 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UUi8yLS8WRGgMAStgNPYL5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZrN2YTA352fRE3dKKB9n5d | Simulation Run 2026-08-16 21:10 | no transcript | — | |
coSizWDz6kzXAHa8mazEBE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ed5kSmGi3tdztEX2XpvkUp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
j4V8b8VXJTtMYkDbJRHny4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nyne3rtqxr9p8YY7EVexch | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dERbdXCctnF3oPqf4HRacV | Simulation Run 2026-08-18 04:44 | no transcript | — | |
CDhQaSU7svUQcZgEmsERWY | Simulation Run 2026-08-18 04:51 | no transcript | — | |
RFcRiL9jt56399LYCbktjF | Simulation Run 2026-08-18 04:51 | no transcript | — | |
AvmhknZsiqEvKByKKS7Hfh | Simulation Run 2026-08-18 23:57 | unjudged | 1x 5 Piece Chicken Tenders (Blackened Ranch, Classic); | |
HavnS8F6ZDRC5B7tJd54W7 | Test new prompt (Aug) | no transcript | — | |
QxwdgphNFQRF4SFWuKjTbN | Test new prompt (Aug) | no transcript | — | |
hwhgo4cN3UuB2fSwAdUKxi | Test new prompt (Aug) | UNKNOWN | The transcript ends before the agent completes the order or directs the customer to pull forward. | empty |
kyJHpy27Nb8WqJe2Wij3g7 | Test new prompt (Aug) | no transcript | — | |
h22Togn8vPtSozrD2xMMFV | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent took the full order, confirmed it twice, and directed the customer to pull forward with no escalation. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
eNw2P837q62JDzKtDGanFG | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 5 Piece Chicken Tenders (Blackened Ranch, Classic); |
kCXcR7mzhAUBX8VTtNQKKZ | Sandbox (Main) | YES | The agent took the full order including all items and sauces and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Blackened Ranch, Classic); |
YJtypY4USyiWabHZ7wPuwS | Simulation Run 2026-08-17 02:14 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Blackened Ranch, Classic); |
WH25C8XdscQ3gXE69VhMrW | Simulation Run 2026-08-18 01:29 | YES | Agent took full order, confirmed it, and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
kerpNbjsmLbiTXxDErqJy9 | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalating to a human. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
B7WWtaDQLiTDDSB9KX6d5c | Simulation Run 2026-08-18 02:10 | YES | The agent took the full order and directed the customer to pull forward without escalating. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B |
JrvKLSSisq2Sd7acmgXgJT | Simulation Run 2026-08-18 05:01 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | empty |
MeCkR5poHVHTXR5MXN7jzN | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to the next window without escalation. | empty |
cMGxdDtcYqzndoieEuSHxi | Simulation Run 2026-08-18 23:07 | YES | Agent fully captured the order and directed the customer to pull forward without escalating to a human. | 1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, S |
CfsUqB9NQr76E2VfCBXL2Y | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
ZHe7zQvLzBxyrEYN8gJGKa | Simulation Run 2026-08-18 23:27 | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
I'd like the 3-piece signature chicken combo, spicy. I don't want fries. What other sides do you have?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
PcqdumCD7ebHqzQX8RnxCM | RBI Bakeoff 2026-07-23 | NO | The agent escalated to a human team member with 'An employee will assist you.' | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
9YRkLvHYRBJFngarQgcUUq | RBI Bakeoff 2026-07-24 Pass 5 | NO | The agent escalated to a human by saying 'An employee will assist you.' after order confirmation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
EtZgDuXefayU2Yx5bhKArC | RBI Bakeoff 2026-07-24 | NO | Agent escalated to a human team member for order verification before completion. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
Mh4JrSWLRRkWrj2uNeEKz8 | RBI Bakeoff | NO | Agent escalated to a human and did not direct customer to pull forward after taking order. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Red |
HdbpxvPPghx23pJ7Jywfxc | RBI Bakeoff | NO | Agent confirmed order but escalated to human when unable to read order back to customer. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
53rJrHmmgvSnhMYre7ptib | Simulation Run 2026-07-20 23:03 (scratch) | NO | Agent said order was incorrect and said 'I'll get an employee', escalating to a human, before directing to pull forward. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Red |
dwCMSm8Ni64EFBb5TvMeH3 | Simulation Run 2026-08-18 01:29 | NO | The order was not fully captured as modifications included 'No Cajun Fries' but final order shows Cajun Fries instead. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
fzCC2drwX5WJ4mG67q66gN | Simulation Run 2026-08-18 01:29 | NO | Agent did not fully capture the order correctly and the order was inconsistent, and some items were missing. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
6DbBYquRPPsuaVsmo7x98k | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8kq3iNVbsHSy9dnVpTQY4Y | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Brhb656Qj5Z7jXDYE8qtNY | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EJbsivhEATA2qdst9723Kg | Simulation Run 2026-08-16 21:10 | no transcript | — | |
b6m8SCnxU7XmiAPsAskoFH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
cMGBRmd5TRrjfsfAfUWP2b | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fk9wobyCLYPpPhAk9q2BGu | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript does not show the agent completing the full order or directing the customer to pull forward. | empty |
hV8VfNNr29ZykLu73FHgLB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
kJeP6nA62XmEUWkjvwm6Qd | Simulation Run 2026-08-16 21:10 | no transcript | — | |
mGgXG2je3voSsjQsKhZNMJ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
5b7P5gQQaAqQqjCeKiiE2F | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript is incomplete; there is no indication the agent took the full order or directed the customer to pull forward. | empty |
J9NDbzhK4fqBzTYyiuQBXQ | Simulation Run 2026-08-18 04:44 | no transcript | — | |
2mYV9cxrrjdkpWtBF9hUzP | Simulation Run 2026-08-18 04:51 | no transcript | — | |
QwdJWP9tHTHiishKJE87Wd | Simulation Run 2026-08-18 04:51 | no transcript | — | |
DmSpi8hgJUUdBPdwJTPkzU | Simulation Run 2026-08-18 23:57 | unjudged | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash | |
5BHRJpNWtoccm9h5VDKUme | Test new prompt (Aug) | no transcript | — | |
9eMFK7LfCxn7oQWT4bTjhq | Test new prompt (Aug) | no transcript | — | |
ZtEYoMChyW5SCvu3d4GRUo | Test new prompt (Aug) | no transcript | — | |
cDacpBuXwpRsVdterd4Qo3 | Test new prompt (Aug) | no transcript | — | |
kb5GnYYXUmAjWRVZQZih7w | Test new prompt (Aug) | no transcript | — | |
5oLDr6F5M8gB2vFQwReXSj | Sandbox (Main) | YES | Agent fully captured order and instructed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
6ZqCrVwnHkqQ2Xj62RGHw6 | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
d5fWt5AW2fpwm42nYn8KSr | Simulation Run 2026-08-17 02:14 | YES | Agent fully captured order and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
9HCqXA9S3nAvRNzSSRjMeR | Simulation Run 2026-08-18 02:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
UhsdV98YYpKbqR2QaP7UVc | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and told the customer to pull forward without escalation. | empty |
kqb2mmeDF7TdztNwKeZ3WS | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
8EU2khDmchp7TmEJssXQNV | Simulation Run 2026-08-18 23:07 | YES | The agent fully captured the order and directed the customer to pull forward with no escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
DSTUVu8VzdML6f8nGEBpYj | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalating or handing off. | empty |
WtHaRKPFaaAEaFdjFkbYKu | Simulation Run 2026-08-18 23:27 | YES | Agent took full order, confirmed it, gave price, and directed customer to pull forward without escalation. | empty |
2c3AQTY4wsQPvkppUcsGRm | Staging (larger sample size) | YES | The agent fully captured the order and instructed the customer to pull forward without escalations. | empty |
MUmBLh2Y74w5YTRgT3BTVX | Staging (larger sample size) | YES | Agent captured full order, confirmed it, and directed customer to pull forward without escalation. | empty |
Can I get the ghost pepper sandwich combo, swap the fries for red beans & rice, and a Dr Pepper?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
P8bSLwf2ySSbvmrpPUgfNe | RBI Bakeoff 2026-07-23 | NO | The agent escalated the order to a human employee instead of directing the customer to pull forward. The order was handed off. | empty |
NRGX6wbqQeCRa8PXEpVNGM | RBI Bakeoff 2026-07-24 Pass 5 | NO | The agent did not direct the customer to pull forward but escalated to an employee for assistance. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
ePPUvBJRCDr8GdVSx5mcSS | RBI Bakeoff 2026-07-24 | NO | Agent escalated to a human team member instead of directing customer to next window. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
RMz8UL6KWxRKm5wFa7YqBv | RBI Bakeoff | NO | Agent escalated to a human team member by stating an employee will assist the customer repeatedly. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
QgYXeANSjaxZdN4oEurE8T | RBI Bakeoff | NO | Agent escalated to a human team member by saying 'An employee will assist you.' | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
6nFcij6keL6MVkDJbBP8PE | Sandbox (Main) | NO | Agent escalated to a human team member instead of directing the customer to pull forward. | 1x Classic Chicken Sandwich Regular Combo (Red Beans an |
W5SDL4XfrRP3gskVPZ3djw | Simulation Run 2026-07-20 23:03 (scratch) | NO | The agent did not direct the customer to pull forward or the next window but escalated to an employee for assistance. | 1x Classic Chicken Sandwich Regular Combo (Red Beans an |
2LAmFgmnVFAYWoWZL3nKe5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
6F67GZa2Vec8Pdde2u5WUK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NFLLaKCe5UHrLyqBTNHiq9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NwZEkPmooiCbBsuz7pLjU6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RmrvMfZFsMsE8KJrPxYqtu | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RvHVWdPq2w6bT4ajyFBriE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UruCYwQzq87hnCXEexH6g3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
f5juToDG5U9s6wjmyPBkcv | Simulation Run 2026-08-16 21:10 | no transcript | — | |
j8iWpc2x6kdZPcDFbpiNmT | Simulation Run 2026-08-16 21:10 | no transcript | — | |
jDDNuaC6Davj8FpmHzydQS | Simulation Run 2026-08-16 21:10 | no transcript | — | |
29d7EqstftsueJrN4pTbw7 | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the agent capturing the full order or directing the customer to pull forward or the next window. | empty |
anwhzAHXmguMaRbFcYULx4 | Simulation Run 2026-08-18 04:44 | no transcript | — | |
EcGw8fHVMtodM4Pp5SSPVp | Simulation Run 2026-08-18 04:51 | no transcript | — | |
gSBqTxhMBsqP4zKZLVPWt4 | Simulation Run 2026-08-18 04:51 | no transcript | — | |
nai4J3dCUnMLLRqzATUe46 | Simulation Run 2026-08-18 23:57 | unjudged | 1x Spicy Chicken Sandwich Large Combo (Red Beans and Ri | |
2Vo6qmM7ReFX37dJarBegu | Test new prompt (Aug) | no transcript | — | |
3VTLgwLZBWauuPAXwPsEYu | Test new prompt (Aug) | no transcript | — | |
YhEfpSLwHWWzb42GiFHfDy | Test new prompt (Aug) | no transcript | — | |
nCyFnd3skg8qT58NUYPoK4 | Test new prompt (Aug) | no transcript | — | |
nkL9atqrd2DcocC35jZAPg | Test new prompt (Aug) | no transcript | — | |
QQctgyNQ8V6A8ugXrzFAhA | Sandbox (Main) | YES | The agent fully captured the order, resolved the substitution, confirmed with the customer, and directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Red Beans and Ri |
FmSmZ84SHzogCELpkmvzY3 | Simulation Run 2026-08-17 02:14 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
Q4s6RodaaLHHagtiv98JfK | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Red Beans and Ri |
fSFQwPpanvqLEVi3JNn2mB | Simulation Run 2026-08-18 01:29 | YES | The agent took the full order, confirmed modifications, handled substitution, and did not escalate. | 1x Classic Chicken Sandwich Large Combo (Red Beans and |
jb8oYxDEjJBBJb83AozUR8 | Simulation Run 2026-08-18 02:10 | YES | The agent fully captured the order, confirmed it with the customer, then directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
9k9N7TMYY2eXdkT7JAXUro | Simulation Run 2026-08-18 05:01 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | empty |
RpCtJDyFHVve4ogW4JyGhS | Simulation Run 2026-08-18 05:01 | YES | The agent confirmed the full order and directed the customer to pull forward without escalation. | empty |
BbJ9QnRQE5LMiTdhQF7Yp9 | Simulation Run 2026-08-18 23:07 | YES | The agent took the full order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Red Beans and |
P2ZR9vGCSJW7iDaXsNrtCv | Simulation Run 2026-08-18 23:19 | YES | The agent captured the full order and directed the customer to pullforward without escalation or handoff. | empty |
Ko7c8nTR6tWBFEJepudWuF | Simulation Run 2026-08-18 23:27 | YES | The agent took the full order and directed the customer to the window without escalating. | empty |
MEy3KC6iJzKFqiCby2Qoev | Staging (larger sample size) | YES | The agent captured the full order and directed the customer to the window without escalation. | empty |
UW6nFpYxdyDahFa4jEfBqc | Staging (larger sample size) | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
One bacon and cheese sandwich combo, large, with fries and strawberry lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
Hp8ZY9dKTZ6bCJq4rtTMhs | RBI Bakeoff 2026-07-24 Pass 5 | NO | The agent handed off to an employee multiple times instead of directing customer to next window. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
jXxexMKwPRZH3gsDWWWpPL | RBI Bakeoff 2026-07-24 | NO | Agent escalated to a human team member instead of sending the customer forward directly. | 1x Classic Bacon & Cheese Chicken Sandwich Large Combo |
kc6zGK3sugeqJWgcy56ijw | RBI Bakeoff | NO | The agent escalated to a human employee for order confirmation multiple times instead of directing the customer to the next window. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
EA8kCDWmqYTZSEkHCB9CkU | RBI Bakeoff | NO | Agent said 'An employee will assist you,' indicating escalation to a human team member. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
Z8PGksrTb7G4LiwCMsf2k4 | Simulation Run 2026-07-20 23:03 (scratch) | NO | Agent escalated to a human team member instead of directing customer forward after full order. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
GF8oM66RdQuMCrUpBBgcuv | Simulation Run 2026-08-18 01:29 | NO | The transcript does not show the agent directing the customer to pull forward or the next window after taking the order. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
VV9MEuYtY6YBinMkmP3mgf | Sandbox (Main) | UNKNOWN | The transcript ends before the agent confirms the full order and directs the customer to pull forward. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
7AHVsugfnHetKvpyToGcoG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
CSxRMfYfQRNQY7ySxQHM8K | Simulation Run 2026-08-16 21:10 | no transcript | — | |
FuuydKKLJ652sJkXzGA38T | Simulation Run 2026-08-16 21:10 | no transcript | — | |
JYzkz5s2gpePkMfNwE33Zt | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TkVk5k7gG9gyJFJmPvNkGp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
VdYb3J2aNq2L32zF9ubeDK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YVBqLAMeN5rEdhMnRoJqB3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dSbUZzxt5RB5vzeQeMp5JK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
mahqsfbo5ENKhXUiNQxFoN | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nbSkEo7RefxpbVaYApivaU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
C88wz4WpwbppuhPPpDtyJn | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the agent completing the order or directing the customer to pull forward. | empty |
SDTYeE5B7ikiKQP25owBzt | Simulation Run 2026-08-18 01:29 | UNKNOWN | Transcript cuts off before order completion or directing customer to next window, no escalation shown. | empty |
8C7gjSrLdNM6ctgG3MdQpD | Simulation Run 2026-08-18 04:44 | no transcript | — | |
4QpkBJJfizffaZrc4px8LB | Simulation Run 2026-08-18 04:51 | no transcript | — | |
5vSU7bvA5FHneBrWm6MwvE | Simulation Run 2026-08-18 04:51 | no transcript | — | |
DjaUq6JYWN67qQmn42fmCP | Simulation Run 2026-08-18 23:57 | unjudged | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (F | |
2KMevboLbRERvAL7qkTfH5 | Test new prompt (Aug) | no transcript | — | |
95fjDVdQzfAVRnfRKjPCzy | Test new prompt (Aug) | no transcript | — | |
R7QSE9S5MxUwxMJZvwWsbZ | Test new prompt (Aug) | no transcript | — | |
dkbTD6AXuajZedDoNvXA3r | Test new prompt (Aug) | no transcript | — | |
hvZjb4qRf3JEWvHDKGcZLG | Test new prompt (Aug) | no transcript | — | |
Xd2xdWAwX3ktTHYejob4Wm | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order and told the customer to pull forward without escalation. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
93Lmz9V3P2RmMdPECiB2my | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
Wde3oYcYNCTahCmQjQVFAs | Simulation Run 2026-08-17 02:14 | YES | The agent took the full order including sandwich, sides, and drink, then instructed the customer to pull forward without escalation. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
m3sbwPt8yyj76sixjfzhG3 | Simulation Run 2026-08-18 02:10 | YES | Agent captured the full order including specifics and instructed the customer to pull forward without escalation. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
9QKSagvrrh278yhGx6sBJv | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
FP9XKP9a2YRLhWxZHYWT95 | Simulation Run 2026-08-18 05:01 | YES | Agent fully captured the order and directed customer to pull forward without escalation. | empty |
D77NDB88nPLkkXgH2eYw3o | Simulation Run 2026-08-18 23:07 | YES | Agent fully captured order and directed customer to pull forward with no escalation or handoff. | 1x Spicy Bacon & Cheese Chicken Sandwich Large Combo (C |
g8DJdDF2LtakqE2vkxSp6y | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
mCc35vP9mW7rjNQ5bzHysL | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
5ahcCjaqNuUExaW3nPKTSd | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
7ZToEa5E8FTEUKnmtHpbNe | Staging (larger sample size) | YES | The agent took the full order and directed the customer to pull forward without escalations. | empty |
Give me the chicken wrap bundle with a mango lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
imhymrrgfp2yXEocswZpFq | RBI Bakeoff 2026-07-23 | NO | Agent handed off to a human employee for order verification and did not direct customer to the next window. | 1x Wraps Bundle (Spicy Wrap, Chilled Mango Cane Sweeeet |
Uf8TgTqzNTzqPm8i85QCTZ | RBI Bakeoff 2026-07-24 Pass 5 | NO | The agent escalated to a human team member to help with order verification. | 1x Classic Wrap Regular Combo |
SteKiA4YFhUTw8q4Pg8WLy | RBI Bakeoff 2026-07-24 | NO | The agent escalated to a human employee to help the customer verify the order. | 1x Wraps Bundle (Classic Wrap, Spicy Wrap, Chilled Mang |
LF3uBCaLVSprakLXAEaZbS | RBI Bakeoff | NO | The agent escalated to a human by saying 'I'll get an employee to help you.' | 1x Classic Wrap Regular Combo (Classic, Chilled Mango C |
gFFKUCk8iBTEBPeB9PkoK2 | Simulation Run 2026-07-20 23:03 (scratch) | NO | The agent escalated to a human team member instead of directing the customer forward. | 1x Classic Wrap |
iK7TyHPeTWJi5rK3F4VhvL | Simulation Run 2026-08-18 23:07 | NO | Agent said they would get a manager and employee to assist, indicating escalation to a human team member. | 1x Wraps Bundle (Classic Wrap, No Blackened Ranch Wrap, |
76rNPL9EeYLBGuGAiDL4JH | Simulation Run 2026-08-16 21:10 | no transcript | — | |
A4BuDarzAWeaQ8wYo6wnKJ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
G4JJsJEm9gVp8EA6AbSPnq | Simulation Run 2026-08-16 21:10 | no transcript | — | |
MbQ5B62MhiVMDk2AxBDxbM | Simulation Run 2026-08-16 21:10 | no transcript | — | |
R9tLSzw5N5AFfNM9KxfLuS | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UwkBvnzG7sUuUijmF9mhM5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZWPNB5nCwjqTE3ZtVmt7XJ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
avzL4jVgaVyGRrFJ3wdvpA | Simulation Run 2026-08-16 21:10 | no transcript | — | |
n3fPhKHWRR494MRxFac84j | Simulation Run 2026-08-16 21:10 | no transcript | — | |
n4CzxXz8QLmVw4LMpfKyWf | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EdBNycVKLGRB5JX6Riyd2B | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the agent completing the order or directing the customer to pull forward or the next window. | empty |
LHnuo7Z4anGSWfXtsp6Hng | Simulation Run 2026-08-18 04:44 | no transcript | — | |
46hAeRGeuyhEwktaumJ7vT | Simulation Run 2026-08-18 04:51 | no transcript | — | |
FzPMEqmKyBTebdAJG8aNe5 | Simulation Run 2026-08-18 04:51 | no transcript | — | |
jJEdrLETLxJRLF2PCfUnAL | Simulation Run 2026-08-18 23:57 | unjudged | 1x Wraps Bundle (Frozen Mango Cane Sweeeet Lemonade, Cl | |
JWHqoyFpUvCM3mEzgbbhbJ | Test new prompt (Aug) | no transcript | — | |
Z5qyn5vHvhCgNujKFVzfgq | Test new prompt (Aug) | no transcript | — | |
fYQ44XSExpHQEbwHf35j93 | Test new prompt (Aug) | no transcript | — | |
gSXnS9ZJDUeDwXyRTbcfaW | Test new prompt (Aug) | no transcript | — | |
nMbJrjEcRDirzBVTUheQZ7 | Test new prompt (Aug) | no transcript | — | |
DB9Ehf8roa57m5gTrf7ETb | RBI Bakeoff | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x Blackened Ranch Sauce; 1x Frozen Mango Cane Sweeeet |
PewFs8zb3DFNnzKEaT3PBX | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Wraps Bundle (Classic Wrap, Spicy Wrap, Chilled Mang |
ciwAeHQ7YFnBX32YBqgpTv | Sandbox (Main) | YES | The agent fully captured the order in detail and directed the customer to pull forward without escalation. | 1x Wraps Bundle (Classic Wrap, Blackened Ranch Wrap, Fr |
VSuSMJ6BnPUUVT8nZPHd9c | Simulation Run 2026-08-17 02:14 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C |
G7gPW6FyKj9AhNXEAMy3EU | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order with specific details, confirmed it with the customer, and directed them to pull forward without escalation. | 1x Wraps Bundle (Classic Wrap, Classic Wrap, Frozen Man |
iJxvf4anfpDn7wUbvVbgaP | Simulation Run 2026-08-18 01:29 | YES | Agent fully captured order details, confirmed with customer, and directed customer to pull forward without escalation. | 1x Wraps Bundle (Spicy Wrap, Classic Wrap, Chilled Mang |
7DnbtWHJZufFBndvgGshZg | Simulation Run 2026-08-18 02:10 | YES | Agent fully captured order details and instructed customer to pull forward without escalation. | 1x Wraps Bundle (Classic Wrap, Classic Wrap, Frozen Man |
7estdjecXJundPfUmsw3SF | Simulation Run 2026-08-18 05:01 | YES | The agent fully took the order, confirmed correctness, and directed the customer to pull forward to the window without escalation. | empty |
hU23gXkj8X96eWpVXgNLSc | Simulation Run 2026-08-18 05:01 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | empty |
fpNGcNWhHAjk9AgEvU7Gki | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
aYirdjB2wCVztqxWLyZXcB | Simulation Run 2026-08-18 23:27 | YES | The agent took the full order and directed the customer to pull forward without escalating or bailing. | empty |
bfrXCPHTv47PHuptzC57Uh | Staging (larger sample size) | YES | The agent took the full order and directed the customer to pull forward, with no escalation involved. | empty |
iX6qvDJKEoth6VDuXxNeLj | Staging (larger sample size) | YES | Agent fully took order and directed customer to pull forward without escalation. | empty |
I'd like the 5-piece tenders combo, blackened and spicy, with cole slaw and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
5iZvibFE8hdVKSEAJS75Kh | RBI Bakeoff 2026-07-23 | NO | Agent escalated the order to a human team member instead of sending the customer forward themselves. | 1x 5 Piece Chicken Tenders Large Combo (Blackened, Cole |
QxJZcw8ZcFGhBPHLHt5q7i | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent said 'An employee will assist you,' signaling escalation to a human after taking the order. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
CXT2vtwwmbCPGYLoHkC65V | RBI Bakeoff | NO | Agent escalated to a human team member to help verify the order before payment. | 1x 5 Piece Chicken Tenders Large Combo (Coleslaw, Mashe |
Q3c3eLvKiVkiBbeTrcM8zu | RBI Bakeoff | NO | Agent failed to read back order and escalated to an employee for assistance, not sending customer forward. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
nTqw369hozJ6qNUGJfTPMx | Simulation Run 2026-08-18 02:10 | NO | Agent confirmed order but did not address error about too many selections and proceeded to close order. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
2RYuXELhLUba4thgM54jtR | Simulation Run 2026-08-16 21:10 | no transcript | — | |
4436RZUYrUtUatGrZPPzXS | Simulation Run 2026-08-16 21:10 | no transcript | — | |
CQCrFrXJ47zXEBfHsjAF8V | Simulation Run 2026-08-16 21:10 | no transcript | — | |
KeGSrrraGydJBPtPVMgWa5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
SumEuDJmGgeo7a427c8d9a | Simulation Run 2026-08-16 21:10 | no transcript | — | |
a23boH42ZUR9LYS84NyE65 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
g2SANquUmBmgvCxrJmXpS6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hrdnC3mvDthytdY3tv44Kj | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nrUu7hGxL2jJntkeFm258M | Simulation Run 2026-08-16 21:10 | no transcript | — | |
o3m6aseQTMM29NFmMf7qZ3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
BPABGRhm2dCKeYJFG29yMe | Simulation Run 2026-08-17 01:07 | UNKNOWN | Agent never directed the customer to pull forward or the next window in the transcript. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
gtDzgc6DnMGS5nBHgX8V3e | Simulation Run 2026-08-18 04:44 | no transcript | — | |
R4fDGokDF6iao46kKn7kF9 | Simulation Run 2026-08-18 04:51 | no transcript | — | |
ZcKXDfsH8PVscB2f2xvDbc | Simulation Run 2026-08-18 04:51 | no transcript | — | |
CkvTqFEepPcCqdQXFpXZfE | Simulation Run 2026-08-18 23:57 | unjudged | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co | |
33JHq99shj9QAkRfS9KHpn | Test new prompt (Aug) | no transcript | — | |
KSyb5t9g6s3TASsZaVdDhh | Test new prompt (Aug) | no transcript | — | |
NKEX6JoUhxasJ7WQLB8Ktp | Test new prompt (Aug) | no transcript | — | |
VbahLaAYjHHiGqJ2wCkgpe | Test new prompt (Aug) | no transcript | — | |
fmAdmJCGX85wuRCBxxjp4Z | Test new prompt (Aug) | no transcript | — | |
UHEsokKZH4UxXGvzCgix2e | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
KcpBpASryTi8usq4b4xHmd | Sandbox (Main) | YES | Agent captured full order details and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
TS5xsGw435PnfRx3hvX94x | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
6jqK4zRr5HjJogL4XAfr3m | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent captured full order details and instructed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Coleslaw |
MiytYjX7MET7CbRHdiRjqg | Simulation Run 2026-08-17 02:14 | YES | Agent fully captured order, confirmed it, and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Spicy, Colesl |
TwTmEDiLPLjLgu4BEGPCCN | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
aPzS6YSDKM9FJTZPDgqfch | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
5nbeXwr8YeoBrHohXBXeJZ | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
99aiQeCKeQ4AzKp4ryqk8S | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
UrtmR7y2pAuwVPAJHTckKT | Simulation Run 2026-08-18 23:07 | YES | Agent fully captured order details and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co |
33XjBJs8ad6FLcRdanYQBF | Simulation Run 2026-08-18 23:19 | YES | The agent confirmed the complete order and directed the customer to pull forward without escalation. | empty |
5wDjBpqmkcmnhZc7BYHHxi | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
5oc5MJEtcWkzu2teNoqzEH | Staging (larger sample size) | YES | The agent captured the full order and directed the customer to pull forward without escalating. | empty |
f4T8VRTMbjrxuo6PoAJ4ra | Staging (larger sample size) | YES | The agent captured the full order and directed the customer to pull forward without escalation. | empty |
Order one classic chicken sandwich by itself. Do not ask for a combo. If the agent offers to make it a combo, accept and choose Cajun fries and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
RuTtQrmmLf27c3v4YWyL3b | RBI Bakeoff 2026-07-23 | NO | Agent escalated to a human team member instead of completing order and directing customer forward. | 1x Classic Chicken Sandwich Large Combo (Cajun Fries, L |
UKjchXdhZm4UH3uuNCGWfh | RBI Bakeoff 2026-07-24 Pass 5 | NO | The agent escalated the customer to a human employee instead of directing them to the next window. | 1x Classic Chicken Sandwich Large Combo (Cajun Fries, L |
AoaPp5jeYbGw4XSGvyAD8u | RBI Bakeoff 2026-07-24 | NO | The agent escalated to a human team member instead of directing the customer to pull forward or the next window. | 1x 1 Piece Custom Signature Chicken (Classic); 1x Class |
M6NuVyVe4w9F9VtxUnvJLC | Staging (larger sample size) | NO | Agent threatened to get employee help if order was incorrect, indicating escalation possibility. | empty |
2vLxYaS4jTsB2uwgC6N8x5 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8u7ZxLVgxYHmakESsQnS7z | Simulation Run 2026-08-16 21:10 | no transcript | — | |
9Hegyk2nTmd3L9MD9ziPHL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ASkaB5Bh8s5YrWsdiBAj9J | Simulation Run 2026-08-16 21:10 | no transcript | — | |
B4eWEPssCqkRqPdZReaEZ8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
J3raDZMGmozrZzub5ywbSW | Simulation Run 2026-08-16 21:10 | no transcript | — | |
M63FRMU5SmaD6pbyfYweHC | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QjNmASLWJSqZQFU2ofM53X | Simulation Run 2026-08-16 21:10 | UNKNOWN | The agent did not complete the order or direct the customer to pull forward in the transcript provided. | empty |
agrHmtQRCfnKPr2pjMHQ23 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
heDNLmjRCpNMNyjKBKNNvK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
4EtpUQrgXjsQgr4jfLzX4y | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not show the agent taking the order or directing the customer forward. | empty |
KhMhWRCF7fb4yWYgdhNmko | Simulation Run 2026-08-18 01:29 | UNKNOWN | The transcript ends without the agent directing the customer to pull forward or the next window, nor escalating to a human. | 1x Classic Chicken Sandwich |
XJph946athbyhFDR45rvoY | Simulation Run 2026-08-18 04:44 | no transcript | — | |
EMrvF4WxTo36gAcAC3AYWU | Simulation Run 2026-08-18 04:49 | no transcript | — | |
MgdjwpBWvT9xu9Md2A295Z | Simulation Run 2026-08-18 04:51 | no transcript | — | |
hkeC5cGMmkh8NepMv4EAxS | Simulation Run 2026-08-18 04:51 | no transcript | — | |
2VjhPPAkjyHQdgc3CQwybP | Simulation Run 2026-08-18 23:57 | unjudged | 1x Classic Chicken Sandwich | |
2Nuw9pLWCcVfU9XWdb6BEs | Test new prompt (Aug) | no transcript | — | |
48eck9K7eVz64yubvvBu8V | Test new prompt (Aug) | no transcript | — | |
CMqs22Nivs3ackv8hWs8eU | Test new prompt (Aug) | no transcript | — | |
dW9xvUQkUdjnbH37DVyitj | Test new prompt (Aug) | no transcript | — | |
gGxwVfwcRA5xnR2BUkWYYA | Test new prompt (Aug) | no transcript | — | |
YcLDSvH5KBpHeQs25rGWXM | RBI Bakeoff | YES | Agent fully captured the order and directed the customer to pull forward without escalating. | 1x Classic Chicken Sandwich |
jFnor9XeE3xXcSUKjMfqrP | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich Regular Combo (Cajun Fries, |
XaptGBzwexAqbxyRzqgXN9 | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Classic Chicken Sandwich |
dUUmtTjSp45XcuwQdSDTQ7 | Sandbox (Main) | YES | Agent confirmed full order of one classic chicken sandwich and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
NQ45RABnHy5RzA6VXHRqrh | Simulation Run 2026-07-20 23:03 (scratch) | YES | Agent fully captured the order then directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
7Gk6JBRJnLCmTm7Yf9uete | Simulation Run 2026-08-17 02:14 | YES | The agent took the customer's full order and instructed to pull forward without escalation. | 1x Classic Chicken Sandwich |
69NCFeQvx2zVjbj3uAwq6w | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
fCmVbh87Wi2FT6CjszYJ2x | Simulation Run 2026-08-18 02:10 | YES | Agent fully captured the order and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
JZns2U2LCgQFzfyRkqGc5V | Simulation Run 2026-08-18 05:01 | YES | The agent captured the full order (one classic chicken sandwich) and directed the customer to pull forward without escalation. | empty |
iYiPeprLY6X8tphsFhHLDV | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
RQQpnGh6KG27YUPmkUAJCe | Simulation Run 2026-08-18 23:07 | YES | Agent confirmed full order and directed customer to pull forward without escalating. | 1x Classic Chicken Sandwich |
3t3EeLctTZRejmrxfhzm2Y | Simulation Run 2026-08-18 23:19 | YES | Agent confirmed full order and directed customer to pull forward without escalating or bailing. | empty |
SbXZkobmSNcEg6N8V3vPCy | Simulation Run 2026-08-18 23:27 | YES | The agent fully took the order and instructed the customer to pull forward without handing off. | empty |
jxUKHgnyJ6aJRdz6a76Rm2 | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
I'll do the 12-piece family meal with all dark meat, spicy, Cajun fries and cole slaw, and two Sprites.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
o9YSWrseixTAJv4CHSpRs6 | RBI Bakeoff 2026-07-24 | NO | Agent escalated to a human team member instead of directing customer to the next window. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
QmL3kths2eNuEQLtWyrr5s | RBI Bakeoff | NO | The agent said 'An employee will assist you,' indicating an escalation to a human team member. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
ajVXLjSYvZu9Fz4k2uxso4 | RBI Bakeoff | NO | Agent escalated to a human employee instead of directing customer to next window after order. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
B64Mu9SPLBomiNnruQDBVX | Simulation Run 2026-07-20 23:03 (scratch) | NO | Agent failed to read back order and stated an employee would assist, indicating escalation. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
CmbncZpiCktuzEtQd8ZUWx | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Hm4QbmsUhrdt2vZozVy2Pv | Simulation Run 2026-08-16 21:10 | no transcript | — | |
LFv5yy2LCcSNpBDMjY3uE9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
TXzfNpnBFu6pQMTC2GJey3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
VwtuG39bbrZwzL4d62SEda | Simulation Run 2026-08-16 21:10 | no transcript | — | |
XbymCVNQG7JxdriM4P72fL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
XrqBE7qrfF9BmGBQyzDEr8 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
YcxCU5ZPKotTq7qn7jQ4Bb | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ZHMqfNZ4A9NqAw9t5eFZdF | Simulation Run 2026-08-16 21:10 | no transcript | — | |
aUMLhEmyXHTyknAajTq3M6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fL7ai2MBkgUDs9DR3QR6JR | Simulation Run 2026-08-17 01:07 | UNKNOWN | Transcript only shows the customer order; no agent response or direction provided. | empty |
QRUjP83UWdtmfHsLPS8DAw | Simulation Run 2026-08-18 04:44 | no transcript | — | |
GZo4MnyLaBWzVPQxtic9NA | Simulation Run 2026-08-18 04:51 | no transcript | — | |
k6cbjdoNC3HJtCVSTFnJjr | Simulation Run 2026-08-18 04:51 | no transcript | — | |
LSJUjuQ6aD48wJCdoQmJfi | Simulation Run 2026-08-18 23:57 | unjudged | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark | |
7PmBiu95N5xKs9JLYrRhDD | Test new prompt (Aug) | no transcript | — | |
EEfcdgQkJhNAKAzA5YBGDc | Test new prompt (Aug) | no transcript | — | |
Esymti6aZtrSjeJSNfCXky | Test new prompt (Aug) | no transcript | — | |
GnAbZpgbnzzT4tWHxuMbAA | Test new prompt (Aug) | no transcript | — | |
f7wVFtBUrFcHKTXvuEifUx | Test new prompt (Aug) | no transcript | — | |
F538nf5MTQH7AvVb63rchF | RBI Bakeoff 2026-07-23 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
6sUQMpy9cfu5WRaePzf7x7 | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Dark Meat, S |
9LoEPQRYqPJi2QKXBARvT4 | Sandbox (Main) | YES | The agent confirmed the full order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
K3xU8tmQjGXFsvniXrhwwE | Sandbox (Main) | YES | The agent fully captured the order details and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
dQ8wWerAvdqmZjCG8zDAYh | Simulation Run 2026-08-17 02:14 | YES | Agent fully confirmed order details with customer and directed to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
253S6aJ5T9Kk2xDTHRybR4 | Simulation Run 2026-08-18 01:29 | YES | The agent fully confirmed the order and directed the customer to pull forward without escalating. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Large |
hRPLx2vLpfkQHiCKvoSzUZ | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
CWJCppYUURWT6DqqJj9H4h | Simulation Run 2026-08-18 02:10 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
Vzf7AbcGvZajjXzuZXFdEG | Simulation Run 2026-08-18 05:01 | YES | The agent confirmed the full order and directed the customer to the window without escalation. | empty |
mnaqoHdTxrSP9ggUxs6Frt | Simulation Run 2026-08-18 05:01 | YES | The agent confirmed the full order and directed the customer to pull forward without escalation. | empty |
eNXgSyqAgTSqjvKvy4mSP6 | Simulation Run 2026-08-18 23:07 | YES | The agent confirmed the full order and directed the customer to pull forward without escalation. | 1x 12 Piece Signature Chicken Family Meal (Spicy, Dark |
4gzXrXVGnDHWsdFa5J8Dgx | Simulation Run 2026-08-18 23:19 | YES | Agent fully captured order and directed customer to pull forward without escalation. | empty |
47GZyPPM6LLD9mYY3ZCwV6 | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and told the customer to pull forward without escalating. | empty |
7LGUikn5cfLj6B8dBW72qq | Staging (larger sample size) | YES | Agent fully captured order and directed customer to pull forward without escalation or handing off. | empty |
JDQAdRpnWBVw4FKiXENCFg | Staging (larger sample size) | YES | Agent took full order, confirmed it with customer, then directed them to pull forward without escalation. | empty |
I'll take the 3-piece chicken combo, spicy, add a pie - make it the strawberry - and a premium strawberry lemonade.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
WacTafaXqzoAJKNzseaDNZ | RBI Bakeoff 2026-07-23 | NO | Agent escalated to a human employee rather than directing the customer to pull forward or the next window after order completion. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Frozen |
9hKZYye4yrEdJdMXP4mLGd | RBI Bakeoff | NO | The agent escalated to a human team member instead of directing the customer to the next window. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
J58Yi4U6qj3fhkB75YmBhx | Staging (larger sample size) | NO | The agent did not complete the order but escalated to a human employee to verify the order. | empty |
2UdFn7K7dAeYX9bMy9tyxR | Simulation Run 2026-08-16 21:10 | no transcript | — | |
93JEriyuCFm3NaD6LtJsGi | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Aueb4Jx3AHVjfyjJdir9RG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EBmEYMZCUTL8kPzaJjQzdv | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EYGSGVV7GnyAmzYDN2iuvU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PMyokv4ALvBnGV6uXTfnqp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Y86URjkJCVe2bbz9Uov4hi | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ewm4ih2v8o5jpHdW8LZPuk | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fgN2ysXqKLgqZziPbH6qhK | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript does not show the agent confirming the full order or directing the customer to pull forward/the next window. | empty |
nirbP2yqphLMVS6hWHr6kG | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript ends before order completion or directing customer to pull forward is shown. | empty |
HW4rMkVymVUGCrZ6knQQuo | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript lacks any agent responses or direction to the customer about pulling forward or next steps. | empty |
echX8MP44qzSNK2FrjXj72 | Simulation Run 2026-08-18 04:44 | no transcript | — | |
3gi2ZGZqnVwMB9tH2qyWbs | Simulation Run 2026-08-18 04:51 | no transcript | — | |
4AytDBxmRES4PWD99CMkSR | Simulation Run 2026-08-18 04:51 | no transcript | — | |
gcWXBrgLFDrGMbM2HWHso8 | Simulation Run 2026-08-18 23:57 | unjudged | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash | |
EGfVsixmq2BEVcvWmW64pZ | Test new prompt (Aug) | no transcript | — | |
KS9Xvy8ozSGdkFX4apgdQe | Test new prompt (Aug) | no transcript | — | |
MGrbcMSBf5g3pHCQxT2gT3 | Test new prompt (Aug) | no transcript | — | |
iriw4xx8xcQ9EnPJC7DH8s | Test new prompt (Aug) | no transcript | — | |
niBaxyL7ibjQhModfGmJPL | Test new prompt (Aug) | no transcript | — | |
Qz3YNkXENYXUWgXDpW4gWq | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order, confirmed all items multiple times, and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil |
2Ab5uF7oMyQNAUfTCWxsdw | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
AZfYrsgDNVUFXnv5dryAaX | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
T4uMWojGYkpTWVyM7kPqMa | Sandbox (Main) | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil |
Ug9X6oKYtEw7z6gGbLBVDG | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil |
MHZGTFgsyWrTHL3V3YFkdj | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent took the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Froz |
eW6PHLJbjDVexDeKWWZyRh | Simulation Run 2026-08-17 02:14 | YES | Agent took the full order, confirmed it with the customer, and directed them to pull forward without escalating. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
5Vm5QaXGHb5FRSVtjNRQMp | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the customer's detailed order and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
VxiPYzRBeA8rn553KxtbNu | Simulation Run 2026-08-18 01:29 | YES | The agent captured the full order accurately and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Caju |
atVAFBAqSN4GATRcVo8Jix | Simulation Run 2026-08-18 02:10 | YES | Agent fully captured the order, confirmed it, and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil |
afvp53Pqvm6PkSyUYEqmFB | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order, confirmed it, and directed the customer to pull forward without escalation. | empty |
i5wGt6Ud6s5BJJpisEDCv2 | Simulation Run 2026-08-18 05:01 | YES | The agent took the full order and directed the customer to pull forward without escalating. | empty |
k3AB8trxx6dzSN584KSLiB | Simulation Run 2026-08-18 23:07 | YES | The agent fully captured the order and then directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
7WVT47MLaLTDHDEykhaYYf | Simulation Run 2026-08-18 23:19 | YES | Agent took the full order and directed customer to pull forward without escalating. | empty |
TNCV3vsYixneNyxpCYdRDt | Simulation Run 2026-08-18 23:27 | YES | The agent captured the full order and directed the customer to the window without escalation. | empty |
eNizBRkvMbnbgJX4jV8VHu | Staging (larger sample size) | YES | The agent took the full order, confirmed it, and told the customer to pull forward without escalation. | empty |
I'll do the chicken sandwich by itself and a side of fries.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
eaTcpw3wqYbZCRM4GZi6ka | RBI Bakeoff 2026-07-23 | NO | Agent escalated to a human by saying an employee will assist customer, failing to complete order and direct forward. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
gkr7vyP2V7yPeGujgvFoyP | Simulation Run 2026-07-20 23:03 (scratch) | NO | Agent escalated to a human by stating 'An employee will assist you.' | 1x Classic Chicken Sandwich; 1x Cajun Fries |
dwmkKCJqMZQhqtmr4pm4rE | Simulation Run 2026-08-17 01:07 | NO | The transcript does not show the agent directing the customer to pull forward or to the next window after taking the full order. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
8b4kNwu5M937EftFEVfvyW | Simulation Run 2026-08-16 21:10 | no transcript | — | |
FZs7JbPbbKQwS7ZAY2KjAh | Simulation Run 2026-08-16 21:10 | no transcript | — | |
J59YQ2suUkNnuoYcB6ZABM | Simulation Run 2026-08-16 21:10 | no transcript | — | |
K4jWV7T7ENbUYxLZhkaU94 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PUasGCaZJzcJ8nNXiPuDX7 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
QQzUStpdUFfNERFQaX9NTk | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UmnMpVfc2DkFpnLsnExmF9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
c3VFowTtPp7oBKfbMjYUDt | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ekPVFFGuFDmnTyXXeAWk8H | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hGy6E565hfbmfGvAX72fUw | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eHDdgNwWXbN7PHqrRFnyhZ | Simulation Run 2026-08-18 04:44 | no transcript | — | |
LfEFSg2DK7645anVNjN7CP | Simulation Run 2026-08-18 04:51 | no transcript | — | |
fVECW2cxm5uFoUB5ddBabU | Simulation Run 2026-08-18 04:51 | no transcript | — | |
WjW5edhViwoZVyRcfiy5XL | Simulation Run 2026-08-18 23:57 | unjudged | 1x Classic Chicken Sandwich; 1x Cajun Fries | |
75CHNe9qr2EErNW3LU3KZT | Test new prompt (Aug) | no transcript | — | |
7YVVjhJmxvGfRpbfjorU4g | Test new prompt (Aug) | no transcript | — | |
8ynA8ecjYMrSF4qy6SUSQm | Test new prompt (Aug) | no transcript | — | |
cEWjinpz7WnxJW7hDtHo2L | Test new prompt (Aug) | no transcript | — | |
kGKG82RWc7b8dsTsDoa7Wq | Test new prompt (Aug) | no transcript | — | |
EYWAgg6LcxeC8EbZ9rKeb7 | RBI Bakeoff 2026-07-24 Pass 5 | YES | Agent captured the full order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries) |
WnUnANFHUw5cWeDkDauCfX | RBI Bakeoff 2026-07-24 | YES | Agent captured full order and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
ZepTZmJKZZ2V5B2efZDjJa | RBI Bakeoff | YES | Agent fully captured the order and directed the customer to pull forward without escalating. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
6xJAgK5aVAykpk6RBm2Mr5 | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
WYfVK72tCpJr8NJdraTAvf | Sandbox (Main) | YES | Agent took the full order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
h2Yj6S4vLURCoiz7nHGd5J | Sandbox (Main) | YES | Agent confirmed full order and directed customer to pull forward without escalation or handoff. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
AdMPt2yaf5p8wY8rRT3FTf | Simulation Run 2026-08-17 02:14 | YES | Agent took the full order, confirmed it, and directed the customer to pull forward without escalation. | 1x Cajun Fries; 1x Spicy Chicken Sandwich |
KrUgTiL5Y6koiaiXQ6oxF4 | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the spicy chicken sandwich and fries order and directed the customer to pull forward without escalation. | 1x Cajun Fries; 1x Spicy Chicken Sandwich |
Z4XRJg9ZcS8nNHbXXh9Yt7 | Simulation Run 2026-08-18 01:29 | YES | Agent took full order, confirmed it, and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
mkTPvM3cBWUDjCwqJ9CLrR | Simulation Run 2026-08-18 02:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
Kd7w2woDsxRgnMNpNzKdi3 | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
QA8gYiBUuTWzixWBzniSVb | Simulation Run 2026-08-18 05:01 | YES | The agent took the full order and directed the customer to move forward without escalation. | empty |
VGszsHsd2atjVnEt32JHPk | Simulation Run 2026-08-18 23:07 | YES | The agent confirmed the full order and directed the customer to pull forward without escalating or handing off. | 1x Classic Chicken Sandwich; 1x Cajun Fries |
kDMVvBJdSdvwEGciZMdfEh | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
MVb8rdKzfWKwoDeG5ZaPtq | Simulation Run 2026-08-18 23:27 | YES | Agent took full order and directed to pull forward without any escalation. | empty |
5Xn7hGpi9ojoAyRB78GAyS | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to the window without escalation. | empty |
c7WQDJFZqHf4DuWfGyuvBg | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
One Ghost Pepper Sandwich, no combo.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
2VnSDr8hjGzG52bK7GbNus | RBI Bakeoff | NO | The agent did not direct the customer to pull forward or the next window, instead said an employee will assist you (escalation). | empty |
FoZpDqPjUDEEP52h3TdVT7 | RBI Bakeoff | NO | The agent did not complete the order or direct the customer to the next window; they escalated by saying an employee will assist. | empty |
TzVVYdmaMBkFjAH7PPC5gU | Simulation Run 2026-08-17 01:07 | NO | Agent confirms full order but then states an employee will assist at the window, indicating escalation. | 1x Spicy Chicken Sandwich |
BZD9eDejEMDprY9MKc3rbB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
LMk95on7PSrs9QWL9ddCdU | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NMoxYiLDrPaLTHLi68T2D9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NkATmEkVZ5yN5vhcVLDGMk | Simulation Run 2026-08-16 21:10 | no transcript | — | |
P5WDx33NMkeBwZViiNW5jL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
RKLB2Vwzz5r5LwjLeUeVo2 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
VDZeFprGm9QnWw3o32ePCB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
WrjAzit9XcbPrfFyqLTDKB | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eDSxeMFxzqPkLTEd9FyP4k | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hhs2VneWvDDW93qHqTBY6s | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fE2oChsPkHRW2eMiMsjGSh | Simulation Run 2026-08-18 04:51 | no transcript | — | |
kLvqJ3PCtLnVSJBinwS8LQ | Simulation Run 2026-08-18 04:51 | no transcript | — | |
U4nUJ83kxQDoPTmCdAU74y | Simulation Run 2026-08-18 23:57 | unjudged | 1x Spicy Chicken Sandwich | |
7TG4H62bJB4iCRTbd69sDE | Test new prompt (Aug) | no transcript | — | |
HqQCCW9K33poPPq3LiiC3W | Test new prompt (Aug) | no transcript | — | |
M3UEpE8M7LpeGD9gDoMhpt | Test new prompt (Aug) | no transcript | — | |
Pjm2R6W3T7kHrXB3Mo7iHc | Test new prompt (Aug) | no transcript | — | |
nQ6BMejpYpxpyLRVfZ9hxj | Test new prompt (Aug) | no transcript | — | |
JvaMeHfQhSq5eUANsrh2WX | RBI Bakeoff 2026-07-23 | YES | Agent took full order of classic chicken sandwich and directed customer to pull forward without escalation. | 1x Classic Chicken Sandwich |
af5kP4HPc7jXEiiw8Aarzs | RBI Bakeoff 2026-07-24 Pass 5 | YES | Agent captured full order and directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
4uBVs6QZMLrZgvsQvr9Vns | RBI Bakeoff 2026-07-24 | YES | The agent fully captured the order for the spicy chicken sandwich and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
3QtkViDP8D7SGcRP7YsL4r | Sandbox (Main) | YES | The agent took the full order for one spicy chicken sandwich and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich |
biA7pjfWpyRUcsfMfNLzQV | Sandbox (Main) | YES | The agent fully captured the spicy chicken sandwich order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
dfEd8jVZddEBGqQKzm3qeB | Simulation Run 2026-07-20 23:03 (scratch) | YES | Agent captured the full order (Spicy Chicken Sandwich) and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
iLiMRfgBpigLZMggwVhgPc | Simulation Run 2026-08-17 02:14 | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
K9ti3dCcjEGXYJauPxiEcD | Simulation Run 2026-08-18 01:29 | YES | Agent fully captured order and directed customer to pull forward without escalating to a human. | 1x Spicy Chicken Sandwich |
VaUYq4EGxKAKv8n6AQUmi7 | Simulation Run 2026-08-18 01:29 | YES | The agent took the full order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
4HX3ZYgDLzr7dhPhVeyP2g | Simulation Run 2026-08-18 02:10 | YES | Agent took full order of one Ghost Pepper Sandwich and directed customer to pull forward without escalation. | empty |
dNZ9nde9HuxkTZr5CLB3zZ | Simulation Run 2026-08-18 04:44 | YES | Agent confirmed full order and directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich |
DcsuKqpZFDMbTxXNXevPVb | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to move forward without escalation. | empty |
Rx3m3BsfssDtwso6Kun8ko | Simulation Run 2026-08-18 05:01 | YES | The agent confirmed the full order and directed the customer to proceed forward without escalation. | empty |
LUmTziP82DAsTZsvRqZ8jj | Simulation Run 2026-08-18 23:07 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich |
ML6Pr6JLifHq5UJW7MzPvH | Simulation Run 2026-08-18 23:19 | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
LFeHYuchjxGFBzrywedCsb | Simulation Run 2026-08-18 23:27 | YES | Agent fully captured the order and directed customer to move forward without escalation. | empty |
A7epd3TurayNFfgd2mkmDy | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
Ha7kgGUXDYTRiCmrPfVxoC | Staging (larger sample size) | YES | Agent confirmed order fully and instructed customer to pull forward without escalation. | empty |
Order one regular bacon and cheese sandwich by itself. Do not ask for a combo or a large. If the agent first offers to make it a regular combo, accept with fries and a Coke, then accept a subsequent l
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
JWpHuC8G9f7gYG7y3Ct7vt | RBI Bakeoff 2026-07-23 | NO | Agent escalated to an employee instead of directing customer to next window. | 1x Bacon & Cheese Chicken Sandwich Dinner |
maH8mLvrMVCpHFY5tzppQv | RBI Bakeoff 2026-07-24 | NO | Agent ended by saying 'An employee will assist you', indicating escalation to a human team member. | 1x Classic Bacon & Cheese Chicken Sandwich |
A7nhVHUSUEdk3aLMVcE3yP | Simulation Run 2026-07-20 23:03 (scratch) | NO | Agent confirmed order but repeatedly directed customer to an employee, indicating escalation to a human. | 1x Classic Bacon & Cheese Chicken Sandwich |
2uS3hfx74eGg5xvL2SPgJA | Simulation Run 2026-08-16 21:10 | no transcript | — | |
54GxQyz7deGrdbCycPzrUK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ELCLpeyaQSzhYeFo8pGq23 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
GHMzXW9hfoDcfZ3RK9zRNY | Simulation Run 2026-08-16 21:10 | no transcript | — | |
MhFarD2JbostVggqd4fPue | Simulation Run 2026-08-16 21:10 | no transcript | — | |
NewVkDaVGQDhQn6dPnWc5z | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Z6RhqCaxwtQwb2eSK8woXj | Simulation Run 2026-08-16 21:10 | no transcript | — | |
aYdsrz8QeMYb6ab9hbvY4t | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ipzHFF2vzVeWfTqiBvyLM3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
jYQSGw7RdVbo5g9n5rDPui | Simulation Run 2026-08-16 21:10 | no transcript | — | |
26DDtdhYVAxYAJApmNPRWZ | Simulation Run 2026-08-18 04:51 | no transcript | — | |
a4D7DTmeP5sB33iM4mrhiH | Simulation Run 2026-08-18 04:51 | no transcript | — | |
aaHk6PWx4QRQHKvfa9oi84 | Simulation Run 2026-08-18 23:57 | unjudged | 1x Classic Bacon & Cheese Chicken Sandwich | |
7T4oXAhVSvhkz8VycGWiyT | Test new prompt (Aug) | no transcript | — | |
8QGpspMkqpiHpmLG7tLaW4 | Test new prompt (Aug) | no transcript | — | |
DfuAzYQ8ktfbjpuLBLkkVx | Test new prompt (Aug) | no transcript | — | |
P5pd55YuxdGZXW5UMTCcaF | Test new prompt (Aug) | no transcript | — | |
UDq7tToweefg9NUD6gBhMV | Test new prompt (Aug) | no transcript | — | |
7yM3SeJiVdwsJiF4cHYGFx | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Bacon & Cheese Chicken Sandwich Dinner |
HuN5Gt9MBGXxZbUhnu3QjR | RBI Bakeoff | YES | Agent captured full order of one classic bacon and cheese chicken sandwich and told customer to check the screen without escalating. | 1x Classic Bacon & Cheese Chicken Sandwich |
3XmDTZuAHf6USZpGRWUyPU | RBI Bakeoff | YES | Agent confirmed the full order and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
WoSoocytCsaKutBekUf32T | Sandbox (Main) | YES | Agent took full order and directed customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
atAPqw9X6Jc5cAHHacUEAF | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
FbFnHDGXUJtuNcCzmjTWjj | Simulation Run 2026-08-17 01:07 | YES | Agent fully captured the order for one classic bacon and cheese sandwich and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
PxGMT3nQe5Cywgjzn6Av9G | Simulation Run 2026-08-17 02:14 | YES | Agent fully captured the order and directed customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
G4coCojVQRJCjnnJYtU7h4 | Simulation Run 2026-08-18 01:29 | YES | Agent fully captured order and directed customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
dfzehu3EgCUjQSy4h6GPTq | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
6zyz6V3fLzHTxdfxNCJeT8 | Simulation Run 2026-08-18 02:10 | YES | The agent took the full order and directed the customer to pull forward without escalating. | 1x Classic Bacon & Cheese Chicken Sandwich |
FL4FXuQLCXxSVf6hB4PZn4 | Simulation Run 2026-08-18 04:44 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
SQmwiBTfbNyc9zF7MNjtS6 | Simulation Run 2026-08-18 05:01 | YES | Agent took full order and directed customer to pull forward without escalating or bailing. | empty |
oMwsibSrow5bY5ew3mhDX3 | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to the window without escalation. | empty |
N6LqxtS6ANXhaNTSgpCcKX | Simulation Run 2026-08-18 23:07 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Classic Bacon & Cheese Chicken Sandwich |
naaWPAssPryaRtTqfeqQik | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
PkzmERUzfEPSFczDvVonJF | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
8SzaMsmxoJzRCUobNHhLRH | Staging (larger sample size) | YES | Agent fully captured the sandwich order and directed the customer to pull forward without escalation. | empty |
h2FEjqg5u6TsF92hwiDEWY | Staging (larger sample size) | YES | Agent took full order, confirmed it, and directed customer to pull forward without escalation. | empty |
Order three pieces of spicy signature chicken by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose mashed potatoes and a Coke.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
ccWJNqsxw6iwrtBr8PZxRr | Simulation Run 2026-08-16 21:10 | NO | The agent did not direct the customer to pull forward or to the next window after completing the order. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
h2igiSmwAivKjtkSh4paYC | Simulation Run 2026-08-16 21:10 | NO | The agent asked twice about upgrading or sides but never directed customer to pull forward or next window, nor confirmed order completion or sent forward. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
iLof6KRxtaxRXRTvqKQhSr | Staging (larger sample size) | NO | The agent repeated the initial question mid-conversation, indicating confusion or failure to fully capture the order initially. | empty |
A4nPKPnpSn9yfmkuXiJADc | Simulation Run 2026-08-16 21:10 | no transcript | — | |
EKTL6PZb2szj38YG28sxht | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript does not include the agent directing the customer to pull forward or the next window, nor any indication of order completion or escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
JcZEERa5emVdHCypjysxvA | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript ends before agent directs customer to pull forward / next window or completes order without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Coca |
7aXfVp4NaHXxgU8WKMLM4q | Simulation Run 2026-08-18 04:44 | no transcript | — | |
kcjcP54Aii3ibRq6aXTzVA | Simulation Run 2026-08-18 04:51 | no transcript | — | |
nDcUZWrga6hiCYmum5Z68o | Simulation Run 2026-08-18 04:51 | no transcript | — | |
3CNv6f8jPB7SJSGDXbSRbz | Simulation Run 2026-08-18 23:57 | unjudged | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed | |
Eg8XbcuCGLWqBSM7gE8WwF | Test new prompt (Aug) | UNKNOWN | The transcript ends before the agent confirms the full order or directs the customer to proceed, with no escalation evident. | empty |
GvVsa3YnVC95rLyUkLrh5u | Test new prompt (Aug) | UNKNOWN | Transcript ends before the agent confirms the full order or directs the customer to pull forward/next window. | empty |
QtXuAAosPvYCbMmTh52fpu | Test new prompt (Aug) | UNKNOWN | The transcript ends before the order is fully captured or the customer is directed to pull forward. | empty |
ayhPpMFvzKvk3YWnJ4okTS | Test new prompt (Aug) | UNKNOWN | The transcript ends before the full order is confirmed or the customer is directed to pull forward/next window. | empty |
nVhr4T8BMSWHfzkJBQvgRr | Test new prompt (Aug) | UNKNOWN | The transcript ends before the order is fully captured or the customer is directed to pull forward or the next window. | empty |
PoQ8vqCvbzQoNPQUc5tTK5 | RBI Bakeoff 2026-07-23 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
4k5ReHQgPYbeHZMyk3WRj7 | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken (Spicy) |
bdunCe7H6eWdaHnHnSwkBb | RBI Bakeoff 2026-07-24 | YES | Agent captured full order and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken (Spicy) |
kyugSNhTsYadSLzW45yehq | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
HNuF6hzVa37DNo6NhF6wYG | RBI Bakeoff | YES | The agent took the full order and instructed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
MLiP2dnowDHsmqfTTCZSEr | Sandbox (Main) | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
dvTDhsukdwYChD2ifCKxcS | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken (Spicy) |
hqcFtSDgEmrDHFR2R4Cb7i | Simulation Run 2026-07-20 23:03 (scratch) | YES | Agent captured full order details and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
9tY83P6o95VDDbpLJCMJvZ | Simulation Run 2026-08-16 21:10 | YES | The agent captured the full order (3-piece spicy chicken large combo with sides and drink) and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
Cqjgaevho2h5zD2aZLEkwo | Simulation Run 2026-08-16 21:10 | YES | The agent fully captured the order details and instructed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
SL4rbv6ZUcCMqVTghqDCEj | Simulation Run 2026-08-16 21:10 | YES | Agent fully captured the order including modifications and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
a9UyCuUfRy9F2vESyzKKmt | Simulation Run 2026-08-16 21:10 | YES | The agent took the full order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
o9LydLjPifwwdT97z7jw9X | Simulation Run 2026-08-16 21:10 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
RHYvpm8E7GJpFXqkBDshZf | Simulation Run 2026-08-17 01:07 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
KYKwRCnuhMfHZQRfBPD6ys | Simulation Run 2026-08-17 02:14 | YES | The agent fully captured the order and directed the customer to the next window without escalating to a human. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
GLaKD2bJruGrcas2buPndY | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
KqpYoSd9Adjv53obPJJ4LK | Simulation Run 2026-08-18 01:29 | YES | Agent took full order, confirmed accuracy, and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
Y65tV5cesbQGGKQTNLDMJJ | Simulation Run 2026-08-18 02:10 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
hGXtuxJkeQJHjcydY2oTvL | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
kirEyi5i3BGXeMTPcsa2tH | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order, confirmed it, and directed the customer to pull forward without escalation. | empty |
JKBvCFKpsckdWg3eBai2Pp | Simulation Run 2026-08-18 23:07 | YES | Agent fully captured order, confirmed it, and directed customer to pull forward without escalation. | 1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed |
JXR32oZvgLEjPWB2jqA7wv | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalating or bailing. | empty |
j8GAbfAjMn5J3JpBE6fdRN | Simulation Run 2026-08-18 23:27 | YES | Agent took full order and directed customer to pull forward without escalation. | empty |
P5UuCUa8kqeaz6APsAWW6g | Staging (larger sample size) | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
Start by ordering the 3-piece signature chicken combo, spicy, with mashed potatoes and a Coke. Before the order is finalized, change it to the 4-piece combo.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
Lp9GjC3s76QDdHMvMp6B2j | RBI Bakeoff | NO | The agent escalated to a human team member by stating an employee would assist the customer. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
KQuWjzuDjj2seafpmhKKNN | Simulation Run 2026-08-18 02:10 | NO | The transcript shows the agent took the full order but did not direct the customer to pull forward or the next window before ending the conversation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
64taAA4AMEmzePALt6mfcL | Simulation Run 2026-08-16 21:10 | no transcript | — | |
8HbSwwecLA5hkgtaNjwHGG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
E9LYapEfvLcuxcLpaMZpB4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Muqp4wNGC6GE7epoDe5zup | Simulation Run 2026-08-16 21:10 | UNKNOWN | The transcript does not show the agent confirming the full order or directing the customer to pull forward. | 1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash |
ZR8hc4WH5E3T9XqDM8RpRM | Simulation Run 2026-08-16 21:10 | no transcript | — | |
cLWNgVdCsex8ampXLoedxk | Simulation Run 2026-08-16 21:10 | no transcript | — | |
dkLipSeHUSeuQvT45x5sF9 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eMTzd5xUqSw3SDNExvmjyT | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nfYYk9sBZtCWwdJGyEMXQK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
oG6rJJVPba2y9QbpkjoDqa | Simulation Run 2026-08-16 21:10 | no transcript | — | |
B9UKmCFgn4KZJXXAA5SRZX | Simulation Run 2026-08-18 04:44 | no transcript | — | |
Jutwps3eGciMEUuCxvWKFq | Simulation Run 2026-08-18 04:51 | no transcript | — | |
cBcBRmMtT4dpFB2bRkCS4Q | Simulation Run 2026-08-18 04:51 | no transcript | — | |
5QgEssdueWktVakp9FnG4e | Simulation Run 2026-08-18 23:57 | unjudged | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash | |
59wVZi2wxRWmPgDeyf4Yam | Test new prompt (Aug) | no transcript | — | |
8JJm4chkCW96faQ9MxnMxF | Test new prompt (Aug) | no transcript | — | |
8r2S2SSjBEXs9uUJXCvm2C | Test new prompt (Aug) | no transcript | — | |
NKkrVTrXuzWZZtcHdMXoGh | Test new prompt (Aug) | no transcript | — | |
Z4XMRaSTuQKsAtRBt57yKU | Test new prompt (Aug) | no transcript | — | |
XKHSjpbBNoR6Vcbz3R47eQ | RBI Bakeoff 2026-07-23 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
btukq3VPKKPqAq4jGnzk8J | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
Sexda7WBSfkLHLf6XgJdVk | RBI Bakeoff 2026-07-24 | YES | Agent took full order with modifications and directed customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
d3Ak4osqVhN4nBSDmfXaZq | RBI Bakeoff | YES | The agent took the full order and directed the customer to pull forward without escalating to a human. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
54yMYMUfdZq2cxkcK8g6iv | Sandbox (Main) | YES | Agent captured the full order (4-piece spicy combo, mashed potatoes, Coke) and directed customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
JNbJwAkjMqKJhQzJFkAr46 | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
eeetUxssa8bfRpReLS96KL | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 4 Piece Signature Chicken Large Combo (Spicy, Mashed |
XcSQxzbqdrqDiXTDqd68kB | Simulation Run 2026-08-17 01:07 | YES | The agent fully captured the customer's order and directed them to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
Jx9wYXcsfnwDr69sg2mYx6 | Simulation Run 2026-08-17 02:14 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
Ycmcf2F9zjJiYQt7GNz3ga | Simulation Run 2026-08-18 01:29 | YES | Agent captured full order and directed customer to pull forward without escalation. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
fop2XaZdTR4sWrkV4HCq6w | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
CRogE5ZmVoyr5hFZXnK6LN | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalating or handing off. | empty |
fhn7NSHmxdgUBqqtwzyqDU | Simulation Run 2026-08-18 05:01 | YES | Agent took full order, confirmed it, and directed customer to pull forward without escalation. | empty |
2zGnHNH5zzGAXuRgbLb47H | Simulation Run 2026-08-18 23:07 | YES | The agent took the full order and directed the customer to pull forward without escalating to a human. | 1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash |
2XGQGGeELF9wuv3JCRNsEP | Simulation Run 2026-08-18 23:19 | YES | Agent captured the full order and directed customer to pull forward without escalation. | empty |
Hrem7Qe3v2P9gZoYuxJo9U | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
WQG4WVDNTkWGB7dBQ6aQ4F | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
e3NigE4JrArEx6LB9FarCv | Staging (larger sample size) | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
Can I get the Ghost Pepper Sandwich combo? Actually, change that to the regular chicken sandwich. And make it spicy.
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
hBf6GkghocFPSKzrA4Ezss | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent did not direct customer to pull forward or next window; instead, agent said an employee will assist. | 1x Spicy Chicken Sandwich Regular Combo (Frozen Mango C |
37KuRPeijM46rj7ubP7qcd | Simulation Run 2026-08-16 21:10 | no transcript | — | |
52WpgzFUDbeNezC5bqGomk | Simulation Run 2026-08-16 21:10 | UNKNOWN | Transcript contains repeated greetings and order requests but no agent confirmation or direction given to customer. | empty |
6QLgH32mNJrwc8uzKKKwBS | Simulation Run 2026-08-16 21:10 | no transcript | — | |
BR4wKDQnVMESUh3bGN3kWe | Simulation Run 2026-08-16 21:10 | no transcript | — | |
UDjcPL8FBAiSTpSC9KQ7P3 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
XyyRxJdYoQStzMKMk8X9dG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
b5CKiYhqkFEa3Rrxqym8WV | Simulation Run 2026-08-16 21:10 | no transcript | — | |
chnEnmbPr2L5bFMLCks3xe | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fxQkSnhiP6GhcnEUQg7vYK | Simulation Run 2026-08-16 21:10 | no transcript | — | |
hNUCuVNRVYNzHhQCYNwvrQ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
GqwPUaHqS7SmDTK4mzEeva | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript shows the customer requesting an order change but lacks agent responses or confirmation of order completion or direction to pull forward. | empty |
3fF4P4a3gZtVVo8foD3ozG | Simulation Run 2026-08-18 04:44 | no transcript | — | |
E4qx9nfNeXAFGumTqdyobu | Simulation Run 2026-08-18 04:51 | no transcript | — | |
N5LzfBV36U6WH2YyRrAQho | Simulation Run 2026-08-18 04:51 | no transcript | — | |
AYeDjP2tTyJ5gJng9caWBN | Simulation Run 2026-08-18 23:57 | unjudged | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C | |
BtCLkibi9Rgx7nkszo4rgK | Test new prompt (Aug) | no transcript | — | |
JPx9rrBtDuGKZ2d8yugwCN | Test new prompt (Aug) | no transcript | — | |
KLcDG3Q2Q492yC7yErDbYB | Test new prompt (Aug) | UNKNOWN | Transcript does not show the agent completing the order or directing the customer forward. | empty |
Tb622WCvTGg4nmRgMd9D8k | Test new prompt (Aug) | UNKNOWN | The transcript does not show if the agent took the full order or directed the customer forward without escalating. | empty |
mFq3m8p8oRYozRU2ULNcrC | Test new prompt (Aug) | no transcript | — | |
EhrWL6MEBEBkkQLBsKKnRU | RBI Bakeoff 2026-07-23 | YES | Agent fully captured the order and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, S |
5E43DJKAUSrERVoieihTKw | RBI Bakeoff 2026-07-24 | YES | The agent took the full order (spicy chicken sandwich combo, fries, sweet tea) and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, U |
Pz6TD5nUbzMSoGmPpMoHcs | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C |
nn2FH6pSiHUTJLjHVPFkXt | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C |
HVHDUREWWEXusCCcMF8nPE | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Mas |
cVfBxNNzBrfhjg4Uo7YjE5 | Sandbox (Main) | YES | Agent fully captured the order and directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C |
HrbnBFxCRPBDRunRnXjbvj | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent fully captured the customer's order and directed them to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C |
VErNyxPLsbUnNU9PduJiQb | Simulation Run 2026-08-17 02:14 | YES | Agent captured full order and instructed customer to pull forward without escalating. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, S |
QWvKwUob6cGAvt7Dzfh567 | Simulation Run 2026-08-18 01:29 | YES | Agent fully captured order and directed customer to pull forward the window without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, S |
QchERGFGXfuCZV3w95cJib | Simulation Run 2026-08-18 01:29 | YES | Agent fully captured the order with fries and Sprite, then directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, S |
e39ZYSpaQzMtFFiAuqmdUu | Simulation Run 2026-08-18 02:10 | YES | The agent fully captured the order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x Spicy Chicken Sandwich Large Combo (Homestyle Mac an |
EbgRnrRdnsH6TxF6sHC26W | Simulation Run 2026-08-18 05:01 | YES | The agent took the full order and directed the customer to pull forward without escalation. | empty |
VhA7L4KhqucyjokT6bHh33 | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
FzMWq8T3bvfpSk9k5eEC5z | Simulation Run 2026-08-18 23:07 | YES | Agent captured full order accurately and directed customer to pull forward without escalation. | 1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, S |
dEpayQtAJ98AU4mvQLDNAK | Simulation Run 2026-08-18 23:19 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | empty |
BxY9iAq5hB2aunpRxXNVGh | Simulation Run 2026-08-18 23:27 | YES | Agent captured full order and directed customer to pull forward without escalation or handoff. | empty |
LJP3Sd5hMAgxzSoMXan32G | Staging (larger sample size) | YES | The agent fully captured the order and instructed the customer to pull forward without escalation. | empty |
gYSDCu6gw6R9YqKbcWRpxQ | Staging (larger sample size) | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | empty |
Do you have nuggets?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
nzyGRsFh5cUaT4VrSEPabb | RBI Bakeoff 2026-07-24 Pass 5 | NO | Agent failed to read back the order and escalated to a human team member for assistance. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Cajun Fr |
HDZMpXUE2VBY8FSF7jXtWp | Simulation Run 2026-08-16 21:10 | no transcript | — | |
Lmn3iJUqkex4XrgJBcdk5Y | Simulation Run 2026-08-16 21:10 | no transcript | — | |
M5NfGZk5vPDRihbpbLJkH4 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
SFCZ2RyQr555osNeJGMdPj | Simulation Run 2026-08-16 21:10 | no transcript | — | |
VJb5eQA43a4SvrSffLvKsr | Simulation Run 2026-08-16 21:10 | no transcript | — | |
akiBThSr7sVyFvc2o8Gz4F | Simulation Run 2026-08-16 21:10 | no transcript | — | |
bpqLngfjndZvWn8YD7ekEZ | Simulation Run 2026-08-16 21:10 | no transcript | — | |
eLB3NPy9RPbBDCATHJxQoz | Simulation Run 2026-08-16 21:10 | no transcript | — | |
fS5Nq2yVGjgTnoVobFD8WF | Simulation Run 2026-08-16 21:10 | no transcript | — | |
gKTFzJ6gkskw9VzRZwVxyE | Simulation Run 2026-08-16 21:10 | no transcript | — | |
McFBEXSkgW5LzhtCBARPH8 | Simulation Run 2026-08-17 01:07 | UNKNOWN | The transcript does not contain any agent response or order process information. | empty |
QYt76kp5t5mQ9ucaGYxSzt | Simulation Run 2026-08-18 04:44 | no transcript | — | |
NXPLLHHw3LhCZy3NzaTAoQ | Simulation Run 2026-08-18 04:51 | no transcript | — | |
PfufDHUKBNnr5YGuecZecB | Simulation Run 2026-08-18 04:51 | no transcript | — | |
4BJZnHscz5X897JSnoLjNz | Simulation Run 2026-08-18 23:57 | unjudged | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Bold BBQ | |
3wFGrugj9xzXqCJn8V98TL | Test new prompt (Aug) | no transcript | — | |
84oXfbFHV5EGuJXK2hizgZ | Test new prompt (Aug) | no transcript | — | |
9UruuTfNjwWgEyS3cTPFJZ | Test new prompt (Aug) | no transcript | — | |
EMRPw5Yz7zmH937BJ9stGb | Test new prompt (Aug) | no transcript | — | |
J3P3ihNc9XGmHz7X9pTVXE | Test new prompt (Aug) | UNKNOWN | The transcript does not contain any order placement or direction to pull forward or next window. | empty |
hzKirj5r5ejFFjLhm23txC | RBI Bakeoff 2026-07-23 | YES | The agent captured the full order and directed the customer to pull forward without escalation. | 1x 3 Piece Chicken Tenders Regular Combo (Cajun Fries, |
Xa3RBnwMuGt6ee5X9FthrA | RBI Bakeoff 2026-07-24 | YES | Agent fully took order, confirmed it, and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Cajun Fr |
RkCgrPHJzBFhkPZ36wZBXA | RBI Bakeoff | YES | Agent captured full order details and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Chilled |
Bf4VhSjmL98Y6BMjcAUvKG | RBI Bakeoff | YES | Agent captured full order details and directed customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Cajun Fries, Ma |
TBjo3csUppfMX9zUxVs39g | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Cajun Fr |
TmY9CqnkahtXBFJ2Vo6Adb | Sandbox (Main) | YES | Agent fully captured the order and instructed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Sweet He |
mEe9VGgGdwFBTWsbjmUuWT | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent captured the full order, confirmed it with the customer, and directed them to pull forward without escalation. | 1x 5 Piece Chicken Tenders Large Combo (Spicy, Cajun Fr |
KHCJjYz5QKncM5j7iVqrHH | Simulation Run 2026-08-17 02:14 | YES | Agent took the full order, confirmed it with the customer, and directed to pull forward without escalation. | 1x Kids Tender Meal (Blackened Ranch) |
ctvE6mmKqE9zd4yc6zFcA5 | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | 1x 5 Piece Chicken Tenders Regular Combo (Spicy, Butter |
h6rGdpLj264byeMeXfkZwm | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order details and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Mashed Potato |
CPFnckdDcu4R5hXYpSxmPN | Simulation Run 2026-08-18 02:10 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x 5 Piece Chicken Tenders Regular Combo (Mashed Potato |
YYH7xYbhsN4kQf8LMvWqc8 | Simulation Run 2026-08-18 05:01 | YES | The agent fully captured the order and directed the customer to the window without escalating. | empty |
g9oPEiykX5UL5x98eBZdsW | Simulation Run 2026-08-18 05:01 | YES | Agent took full order, confirmed it with customer, and directed to pull forward without escalation. | empty |
VprvprxTf7CzAuKH7aTcGG | Simulation Run 2026-08-18 23:07 | YES | The agent captured the complete order and directed the customer to pull forward without escalation. | 1x 3 Piece Chicken Tenders Regular Combo (Cajun Fries, |
eJuwQb7pmSH3CEx2oKBqSu | Simulation Run 2026-08-18 23:19 | YES | Agent fully captured the order and directed customer to pull forward without escalation. | empty |
PYzsFoATvguEPHra3CcWge | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to the window without escalation. | empty |
eZjGntjL859z3akedD7uHK | Staging (larger sample size) | YES | Agent took the full order and directed the customer to pull forward without escalating to a human. | empty |
ijvnfNJtbrWRDPY6Mdr6DX | Staging (larger sample size) | YES | The agent captured the full order and directed the customer to pull forward without escalating. | empty |
Can I have a Cajun Rice please?
| Simulation | Batch | Verdict | Judge note | Final cart |
|---|---|---|---|---|
CLBggiuKwfSjienmPt8pnK | Simulation Run 2026-08-17 01:07 | NO | The agent did not confirm the full order or direct the customer to pull forward; the order was incomplete. | empty |
8kWx3whGNkM8qXSwqd95vf | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ANbuwUKxTbG2QDZAW8CgDV | Simulation Run 2026-08-16 21:10 | no transcript | — | |
AQqFnCcR4tFFB3CnoyWFMb | Simulation Run 2026-08-16 21:10 | no transcript | — | |
HxgCFTdgqVH8CiwmFX5Vjc | Simulation Run 2026-08-16 21:10 | no transcript | — | |
KVzcicEJpQrSnZXE9q5fzD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
ey8m6L6FWNjxNnQCebJYMD | Simulation Run 2026-08-16 21:10 | no transcript | — | |
gXWiJ7KdTTSBzkZrYEE6ZG | Simulation Run 2026-08-16 21:10 | no transcript | — | |
iZ8p6VcwiqGjg4YX6iNKxo | Simulation Run 2026-08-16 21:10 | no transcript | — | |
n5tPaxevP6fxSbQvgiRgZ6 | Simulation Run 2026-08-16 21:10 | no transcript | — | |
nTb4f6gNopdyrm4aCyZ9Wq | Simulation Run 2026-08-16 21:10 | no transcript | — | |
PApPsPxtWdNpQh9TbQxqdo | Simulation Run 2026-08-18 04:44 | no transcript | — | |
fnKRu5zDNCuRjydZDU8sTS | Simulation Run 2026-08-18 04:51 | no transcript | — | |
hMRPU4t3AyzVxoaJ2tswF8 | Simulation Run 2026-08-18 04:51 | no transcript | — | |
mFRTLh6bo53KWVMe6Z65NC | Simulation Run 2026-08-18 23:57 | unjudged | 1x Red Beans & Rice (Rice); 1x Hot Sauce | |
4g27Dt6mwaifq3b2X77kGM | Test new prompt (Aug) | no transcript | — | |
9ExoFDiQQdK5NANH5tWa9B | Test new prompt (Aug) | no transcript | — | |
EgxunFtNqnvvgii6hxk2ep | Test new prompt (Aug) | no transcript | — | |
YVd7stmKAZbVZ36cwXTUqB | Test new prompt (Aug) | no transcript | — | |
bH8YkbeGpj2AwvzysFVtih | Test new prompt (Aug) | no transcript | — | |
H7xqAqxANbCSMMMtdyd6Pv | RBI Bakeoff 2026-07-23 | YES | Agent fully captured order of red beans and rice plus Sprite and directed customer to pull forward without escalation. | 1x Red Beans & Rice; 1x Sprite |
fGV8TzvyGwyeNYb2i5pYPZ | RBI Bakeoff 2026-07-24 Pass 5 | YES | The agent fully captured the order (regular red beans and rice) and directed the customer to pull forward without escalation. | 1x Red Beans & Rice |
DBgqQr3VEzdrXdzeHfUDFn | RBI Bakeoff 2026-07-24 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice |
HYvq6BMnbdedMVeQKCUCw8 | RBI Bakeoff | YES | Agent fully captured the order and directed the customer to pull forward without escalating or handing off. | 1x Red Beans & Rice |
ab87t2asMcRWx6kPcCV6wg | RBI Bakeoff | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice |
63iJJbXZHfNzrfqTVRRCxF | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice) |
9a8UDzxWLdspEEfSzbJB9R | Sandbox (Main) | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice) |
Le5VZug3pjFFHahipbuxxu | Simulation Run 2026-07-20 23:03 (scratch) | YES | The agent fully captured the order for regular red beans and rice and directed the customer to pull forward without escalation. | 1x Red Beans & Rice |
TG8CZtcWffG6mDB37rrAQ3 | Simulation Run 2026-08-17 02:14 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice) |
6coQ83bZgQsS4s3GPfNkza | Simulation Run 2026-08-18 01:29 | YES | The agent confirmed the full order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice) |
n9RuKgnd3wdG4CC5eAFCxk | Simulation Run 2026-08-18 01:29 | YES | The agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice |
9mqbsYwHr5Dxc6cRtV7Utv | Simulation Run 2026-08-18 02:10 | YES | Agent fully captured the order and directed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice); 1x Hot Sauce |
cVe6rrPdAKRmaktrRmXXJJ | Simulation Run 2026-08-18 05:01 | YES | The agent took the full order, confirmed it, and directed the customer to pull forward without escalation. | empty |
hDJoEnavurnZmJip3c6qRn | Simulation Run 2026-08-18 05:01 | YES | The agent took the full order (regular red beans and rice) and directed the customer to pull forward without escalating. | empty |
NQkmyf5nvjhzDZa8EJmxY5 | Simulation Run 2026-08-18 23:07 | YES | The agent fully captured the order and instructed the customer to pull forward without escalation. | 1x Red Beans & Rice (Rice) |
MvuhsX2EpXoENQtMnTtY2F | Simulation Run 2026-08-18 23:19 | YES | The agent captured the full order and told customer to pull forward, with no escalation or handoff. | empty |
fLfEr5yCkjDGTeTsZmsHWC | Simulation Run 2026-08-18 23:27 | YES | The agent fully captured the order and directed the customer to pull forward without escalating. | empty |
FnKg5pxHSUcCeFmLZY2Tfe | Staging (larger sample size) | YES | The agent took the full order and directed the customer to pull forward without escalating or handing off. | empty |
NqTmnnRUXnqrnNhTFd93UG | Staging (larger sample size) | YES | The agent fully took the order and directed the customer to pull forward without escalating. | empty |