← RBI Bakeoff: Coval results

Standard Customer

You are a typical customer calling for assistance. You speak clearly at a normal pace and are generally cooperative. You wait for the agent to finish speaking before responding. You ask relevant follow-up questions when needed and provide requested information when asked. You are neither overly patient nor impatient - you expect reasonable service.

Completion rate
75.0%
123/164 judged conversations
Conversations
439
Failed
41
No transcript
256

Completion rate by test case, this persona only

Test case names jump to that scenario's conversations below.

37.5%
3/8
37.5%
3/8
44.4%
4/9
44.4%
4/9
75.0%
6/8
87.5%
7/8
87.5%
7/8
88.9%
8/9
88.9%
8/9
88.9%
8/9
88.9%
8/9
94.4%
17/18

Every conversation, by test case

Failures first within each scenario. Simulation IDs open the transcript in Coval. The judge note is Coval's written reason for the Completion verdict.

RBI connectivity smoke

0/2 passed (0%)View in Coval →

Order one classic chicken sandwich, no combo. Complete the order, then stop speaking when the agent directs you to the window or hands you off to an employee.

SimulationBatchVerdictJudge noteFinal cart
9pixPf5PUajH3jkFSFugAdRBI v2 clean connectivity proof - deepgramNOAgent said they would get an employee to help, indicating escalation to a human team member.1x Classic Chicken Sandwich
eW7WGijjfjDh223HdXx8knRBI v2 connectivity smoke - deepgramNOAgent did not capture a full order and escalated to a human team member for assistance.empty
ZwmB9Sknw3eZqEhYsmwtmLSimulation Run 2026-07-09 21:45unjudgedempty

UAT #29

3/8 passed (38%)View in Coval →

I'll do the 12-piece family meal with all dark meat, spicy, Cajun fries and cole slaw, and two Sprites.

SimulationBatchVerdictJudge noteFinal cart
XdxoMyu8hgJEMy4azMGTBXRBI Bakeoff 2026-07-23NOThe order was not fully captured as the agent only confirmed one Sprite instead of two.2x 12 Piece Signature Chicken Family Meal (Dark Meat, S
KZStunP48WmmRdbd5k7UpzRBI Bakeoff 2026-07-24 Pass 5NOAgent escalated to a human employee instead of directing customer to the next window.1x 12 Piece Signature Chicken Family Meal (Dark Meat, S
HLMmf53ijZkjraDDq6HhB5RBI Bakeoff 2026-07-24NOThe agent did not fully read back the order and escalated to a human team member instead of sending the customer forward.1x 12 Piece Signature Chicken Family Meal (Dark Meat, S
jW5sj4gJVN9RecmoVkFPCzRBI BakeoffNOThe order was not fully captured correctly (Sprite quantity was inconsistent), so full order was not confirmed before asking to pull forward.2x 12 Piece Signature Chicken Family Meal (Dark Meat, S
V2rarner5aQPEKQho4Q4gSSimulation Run 2026-07-20 22:35 (scratch)NOAgent did not complete the order; escalated to human assistance instead of sending forward.2x 12 Piece Signature Chicken Family Meal (Dark Meat, S
hbEmfeoLkLhc3nrWdidFhkRBI Bakeoff 2026-07-24no transcript—
98hphgoD6yq4CcFqMU44UqSimulation Run 2026-08-16 21:10no transcript—
Chh9vwneE6F47nXpJqiHWkSimulation Run 2026-08-16 21:10UNKNOWNThe transcript shows the order was placed but does not indicate if the customer was directed to pull forward or if there was any escalation.empty
EFZogjurdYKHNfmgiwomKkSimulation Run 2026-08-16 21:10no transcript—
Eu9MgDCjbbVpCmPq36U5v2Simulation Run 2026-08-16 21:10no transcript—
GiqsdysPWhF3qtbLo8DsFiSimulation Run 2026-08-16 21:10UNKNOWNThe transcript does not show the agent confirming the full order or directing the customer to pull forward or the next window.empty
L6KEtWwEk9RVRJBJPoswoMSimulation Run 2026-08-16 21:10no transcript—
NsHXtU6EJFMBXmBQuYQWHjSimulation Run 2026-08-16 21:10UNKNOWNThe transcript does not show the agent confirming or directing the customer to pull forward or go to the next window, nor escalation or handoff.empty
SHP8nmThN7VJFyptGASHJzSimulation Run 2026-08-16 21:10no transcript—
h4RA6VttYiTCcnsiBvCEm8Simulation Run 2026-08-16 21:10no transcript—
jUJW4ssUVvmHGvjDRHULPiSimulation Run 2026-08-16 21:10no transcript—
MtWfWJGqu9R4Djw9r3m5qdSimulation Run 2026-08-17 01:07UNKNOWNThe transcript does not show if the agent took the order or directed the customer forward without escalation.empty
8DATYUTrYqnGUgZbUQD5fJTest new prompt (Aug)no transcript—
LZrLNVYVCyLp6xiTdVSLW6Test new prompt (Aug)no transcript—
RrShK3wnVc2u7wUy2Q3CsnTest new prompt (Aug)no transcript—
WVJW53aF6Zn9vu5C3y5VBtTest new prompt (Aug)no transcript—
j7z2bG6T5CXXouF4YRC3gcTest new prompt (Aug)no transcript—
MkvumeukFuUZ6hvLqb2carRBI BakeoffYESAgent took full order, confirmed accuracy, and told customer to pull forward without escalation.2x 12 Piece Signature Chicken Family Meal (Dark Meat, S
AYXyg7Ugy3wcwpS9dGkFJuSimulation Run 2026-08-16 23:02YESThe agent took the full order, confirmed it with the customer, and directed them to pull forward without escalation.1x 12 Piece Signature Chicken Family Meal (Spicy, Dark
eaJceXMmA4JxvNvMzA5MB6Simulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 12 Piece Signature Chicken Family Meal (Spicy, Dark

UAT #9

3/8 passed (38%)View in Coval →

One bacon and cheese sandwich combo, large, with fries and strawberry lemonade.

SimulationBatchVerdictJudge noteFinal cart
gJxbRi8QVVQCm4Zidmc8GARBI Bakeoff 2026-07-23NOAgent escalated to a human team member twice instead of directing customer forward after taking full order.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
P2MFwRsX9AEBeeYnqS2qv4RBI Bakeoff 2026-07-24 Pass 5NOAgent fully captured the order but escalated by saying 'An employee will assist.'1x Classic Bacon & Cheese Chicken Sandwich Large Combo
KNCcRCfSxNog7XbrLfy335RBI Bakeoff 2026-07-24NOAgent escalated to a human after the customer requested to hear the order back.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
FXFj9jg6YsovEeRmiNVFb4RBI BakeoffNOAgent escalated to a human after taking order and didn't direct customer forward themselves.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
ccTAHm7KPQxA9TkErBzCQFRBI BakeoffNOThe agent escalated to a human team member multiple times instead of sending customer forward without handoff.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
AEcTU734K3ct8kNCsZLTKfRBI Bakeoff 2026-07-24no transcript—
2frxw62rfUaT2pjHP6rym5Simulation Run 2026-08-16 21:10no transcript—
APe6Uo6h99Jem3FBxJXmWwSimulation Run 2026-08-16 21:10no transcript—
Bi6FrMa9C8phoRX6sXJt8FSimulation Run 2026-08-16 21:10no transcript—
P4zJ8gwULCpTgxQSZGQ82NSimulation Run 2026-08-16 21:10no transcript—
SW3wb7EpHMTNiJp4Qobx3LSimulation Run 2026-08-16 21:10no transcript—
cyaFKJbimvg8ZF9pMRddAeSimulation Run 2026-08-16 21:10no transcript—
gRgFdLDEi9JPStbj2VzQ4NSimulation Run 2026-08-16 21:10no transcript—
mzw8FoeWsQovGkRrmVEPZZSimulation Run 2026-08-16 21:10no transcript—
ni5gf4kqrL4NZzEWGU4PtESimulation Run 2026-08-16 21:10no transcript—
oHXNNnWqrRBqw6JNmS6EVaSimulation Run 2026-08-16 21:10no transcript—
XPYNJHsBJMo5oXLBskXPqdSimulation Run 2026-08-17 01:07UNKNOWNThe agent did not confirm, take the full order, or direct the customer to pull forward or the next window.empty
5RUb7xNJMB4uFdrqdn4AiZTest new prompt (Aug)no transcript—
7NHTKZJ7T6gjSbSaFCnC4JTest new prompt (Aug)no transcript—
ChKVqjVzw54KqRhLQ5NXdsTest new prompt (Aug)no transcript—
RrKbyWjdZVXgnrwRCJ5LpDTest new prompt (Aug)no transcript—
Ta3RjkdotGhHdFtq7JN5vMTest new prompt (Aug)no transcript—
RJXER925CVk7QahMhdGFctSimulation Run 2026-07-20 22:35 (scratch)YESAgent fully captured the order and directed the customer to pull forward without escalation.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
dUypNxMkPbw8pBqX7aG3dtSimulation Run 2026-08-16 23:02YESThe agent took the full order, confirmed it, and directed the customer to pull forward without escalating.1x Classic Bacon & Cheese Chicken Sandwich Large Combo
Lzei39Cc6sf2RCXnPRD2f8Simulation Run 2026-08-17 23:31YESThe agent fully captured the customer's order and directed them to pull forward without escalation.1x Classic Bacon & Cheese Chicken Sandwich Large Combo

UAT #2

4/9 passed (44%)View in Coval →

Give me the chicken wrap bundle with a mango lemonade.

SimulationBatchVerdictJudge noteFinal cart
i5mQpjfnRyPkHQ5zLQ9RHoRBI Bakeoff 2026-07-24 Pass 5NOAgent failed to read back the full order and escalated to a human team member.1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C
oQStC82y3waiiCcVg3JQZ9RBI Bakeoff 2026-07-24NOAgent escalated to a human team member multiple times and did not direct customer to pull forward/next window.1x Wraps Bundle (Classic Wrap); 1x Chilled Mango Cane S
SsLiSbEL8yUhekVkjwMFd9RBI BakeoffNOThe agent escalated the order confirmation to a human employee multiple times without finalizing the order or directing the customer to pull forward.1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C
5PAw7bbX9yoDumLr86tYJgSimulation Run 2026-07-20 22:35 (scratch)NOAgent escalated to a human team member instead of directing the customer forward.1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C
czxUYoeyZdiCQbKzZ2qywASimulation Run 2026-08-16 23:02NOThe transcript ends before the agent directs the customer to pull forward or next window without escalation.1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C
TD9nXrYFikZ7TRQrTywpfJRBI Bakeoff 2026-07-24no transcript—
2XEXDfLWBvxAtVuNTejc22Simulation Run 2026-08-16 21:10no transcript—
9r9WF5Uh4dGKcP94YHFb37Simulation Run 2026-08-16 21:10no transcript—
CWbcoQLmqBjBqQdT4TcpuuSimulation Run 2026-08-16 21:10no transcript—
HSopMcZCNULfbeENioioZwSimulation Run 2026-08-16 21:10no transcript—
Hr2hamVk8pfswyjGAZXXmDSimulation Run 2026-08-16 21:10no transcript—
K9w3rXWQteNUfxXbJvQriFSimulation Run 2026-08-16 21:10no transcript—
MgtRvVfc7Eobs6MRHFqXjqSimulation Run 2026-08-16 21:10no transcript—
QNjvJ9B7JAWM7Xg3h9UFKPSimulation Run 2026-08-16 21:10no transcript—
azZLXXeudzJgdEGe8AUhAUSimulation Run 2026-08-16 21:10no transcript—
cy27qBhS5CaZefa33rNepQSimulation Run 2026-08-16 21:10no transcript—
mJEoaehyFFMETJpWhfPPycSimulation Run 2026-08-17 01:07UNKNOWNThe transcript contains only the customer's order request with no response from the agent.empty
6gzxH5W2eG5gC4WMuHtmXRTest new prompt (Aug)no transcript—
GcrXchrCRpE9jsM7aXqZs3Test new prompt (Aug)no transcript—
RiQkee5RexC7Tfd7tjPnvoTest new prompt (Aug)no transcript—
SNvFWzSgAiYghHH7cWT66LTest new prompt (Aug)no transcript—
ZXN57amad66Qy5Ed4sGbbnTest new prompt (Aug)no transcript—
hjVTdm8UWN7x6tYdHkXeypRBI Bakeoff 2026-07-23YESThe agent fully captured the order and instructed the customer to pull forward without escalation.1x 12 Piece Chicken Tender Group Bundle (Classic, Black
8KejEa6aNhjBUmG5QwuqjDRBI BakeoffYESThe agent fully captured the order and directed the customer to pull forward, no escalation occurred.empty
266eSENrUVy9KAvnhteRrdSimulation Run 2026-08-13 19:52YESThe agent fully captured the order and directed the customer to pull forward without escalation.empty
RTdzjrq3k6GgjbFqxmFJwzSimulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Wraps Bundle (Chilled Mango Cane Sweeeet Lemonade, C

UAT #61

4/9 passed (44%)View in Coval →

Order one regular bacon and cheese sandwich by itself. Do not ask for a combo or a large. If the agent first offers to make it a regular combo, accept with fries and a Coke, then accept a subsequent l

SimulationBatchVerdictJudge noteFinal cart
eaqaMbvw8fcFS3y9NDj6X4RBI Bakeoff 2026-07-23NOAgent repeatedly says an employee will assist instead of directing the customer to pull forward or next window.1x Bacon & Cheese Chicken Sandwich Dinner
2fwqYYbv6kNoNoxQ6y8QbTRBI Bakeoff 2026-07-24NOAgent confirmed order but escalated to a human instead of directing customer to next window.1x Bacon & Cheese Chicken Sandwich Dinner
jXNQbfvNK4cfb8qQ6y5rncRBI BakeoffNOAgent did not direct customer to pull forward; repeatedly stated 'An employee will assist you,' implying escalation.1x Bacon & Cheese Chicken Sandwich Dinner
YGgSbDVwNtRxXsLaFMEa95RBI BakeoffNOThe agent escalated to a human employee and did not complete the order themselves.1x Bacon & Cheese Chicken Sandwich Dinner
KCWzNpifNNY2D486CctkHaSimulation Run 2026-08-17 01:07NOThe agent did not direct the customer to pull forward or the next window after taking the order.1x Classic Bacon & Cheese Chicken Sandwich
f7faymwAZSPV8UVcwRbWBzRBI Bakeoff 2026-07-24no transcript—
5rAUUm6qPvbjzeemJrAmhwSimulation Run 2026-08-16 21:10no transcript—
EV36cDVgALLrp4dpxk7yBMSimulation Run 2026-08-16 21:10no transcript—
FooKk8LaG7CgYcKvcAbahVSimulation Run 2026-08-16 21:10no transcript—
KXsomrTyPu3fU82yzHR6dCSimulation Run 2026-08-16 21:10no transcript—
NKUcCc6JRdoZrgJp8zLXYVSimulation Run 2026-08-16 21:10no transcript—
NwADQE4nt9wghPfmYb5Ly8Simulation Run 2026-08-16 21:10no transcript—
aGs38dgsye39cky6rAX4ogSimulation Run 2026-08-16 21:10no transcript—
axgqFTx8vSwbRzRBtx2svESimulation Run 2026-08-16 21:10no transcript—
eZw3em5aubwcZfJvDPsWmzSimulation Run 2026-08-16 21:10no transcript—
mRMtniuohmUx5xv6cwB3XrSimulation Run 2026-08-16 21:10no transcript—
4rNPQiA28T6MHfhq3DxVpqTest new prompt (Aug)no transcript—
6gMBDj3sV93JPwY5k8bgB4Test new prompt (Aug)no transcript—
NkkK3UPyZG8Q4KHNqjmiNvTest new prompt (Aug)no transcript—
TYdxTCbDaMV6iNdYNzGETgTest new prompt (Aug)no transcript—
fFE4YJL4fcosiUyyuyydE4Test new prompt (Aug)no transcript—
EDkws78ufTcMP79VjjdwUURBI Bakeoff 2026-07-24 Pass 5YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Bacon & Cheese Chicken Sandwich Dinner
GMa2owz5Xdp9VAyTQgUdtASimulation Run 2026-07-20 22:35 (scratch)YESThe agent took the full order for a classic bacon and cheese sandwich and directed the customer to pull forward without escalating.1x Bacon & Cheese Chicken Sandwich Dinner
KRXT6DXsD9zobdxqVjPpptSimulation Run 2026-08-16 23:02YESAgent took full order and directed customer to pull forward without escalation.1x Classic Bacon & Cheese Chicken Sandwich
BoHAob2RBt2xGnma26MJstSimulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Classic Bacon & Cheese Chicken Sandwich

Combo offer after tenders

6/9 passed (67%)View in Coval →

Order five classic tenders by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose cole slaw and a Sprite.

SimulationBatchVerdictJudge noteFinal cart
FwQcA6knF5XDZEFZM4LYqZRBI Bakeoff 2026-07-23NOAgent failed to fully read back the order and deferred to employee assistance, indicating incomplete order capture.1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R
R7DFgxZrVsHmmwUsZgUsQ4RBI Bakeoff 2026-07-24 Pass 5NOThe agent stated they cannot read back the order and suggested an employee can assist, indicating escalation to a human team member.1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R
NK7jRHpBFGjPjic79EteJHRBI BakeoffNOAgent failed to read back the order and deferred to an employee for assistance.1x 5 Piece Chicken Tenders Dinner (Blackened Ranch, Bol
KDzriJSbouubErtnxDdHEKRBI Bakeoff 2026-07-24no transcript—
ATMug2ghvcAwP5HXTqH7hpSimulation Run 2026-08-16 21:10no transcript—
FDSJt4YvPi995zJX9Z5AtuSimulation Run 2026-08-16 21:10no transcript—
KTgThtKwaTZQWfzxy9CRd7Simulation Run 2026-08-16 21:10no transcript—
RoSQUzvCPkdVZVEdxYjoCxSimulation Run 2026-08-16 21:10no transcript—
SGiX6dLnUPvLVJd5nLEa5cSimulation Run 2026-08-16 21:10no transcript—
TfcQQJrvP8e9GMAve7HTvfSimulation Run 2026-08-16 21:10no transcript—
boMsaavPZbtGQntMqWg62USimulation Run 2026-08-16 21:10no transcript—
cTjxobKmaeroMBYM4zuWneSimulation Run 2026-08-16 21:10UNKNOWNThe transcript does not show the agent capturing the full order or directing the customer to pull forward.empty
dVApq7dEDgNPA8wctaUeyvSimulation Run 2026-08-16 21:10no transcript—
iuSS9qH7qt763kKufzU5XASimulation Run 2026-08-16 21:10no transcript—
CGxHDdfWgL5KKWyyXQ5uQ4Test new prompt (Aug)no transcript—
Nei4ohFbAJ2HyXw3hbpCemTest new prompt (Aug)no transcript—
ePPHm9VSd9FxQYJtA9p3eZTest new prompt (Aug)no transcript—
edJiXmpnmaKpfes4XLzoLyTest new prompt (Aug)no transcript—
iuMUtywd8gFWEUQVZ63vPGTest new prompt (Aug)no transcript—
huQEMUy5e4Cbxx7VsPdTFZRBI Bakeoff 2026-07-24YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R
eJsCqwEqkNL7PoKg8DaZ6NRBI BakeoffYESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B
AGz6DngXqAZsKPbWFp5VBhSimulation Run 2026-07-20 22:35 (scratch)YESAgent captured the full order and directed customer to pull forward without escalating.1x 5 Piece Chicken Tenders Dinner (Classic, Blackened R
6eN9oAbEYPaLQ2ZUD7UpNuSimulation Run 2026-08-16 23:02YESAgent fully captured the order, confirmed it, and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B
CpPdFCsqfaSwwape8gY3cdSimulation Run 2026-08-17 01:07YESThe agent fully captured a 5-piece classic tenders order and initiated pulling forward without escalation.1x 5 Piece Chicken Tenders (Classic)
VateKdN9Vza2KR3CdukTWDSimulation Run 2026-08-17 23:31YESThe agent took the full order and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders (Classic, Blackened Ranch, B

Combo offer after sandwich

6/8 passed (75%)View in Coval →

Order one classic chicken sandwich by itself. Do not ask for a combo. If the agent offers to make it a combo, accept and choose Cajun fries and a Coke.

SimulationBatchVerdictJudge noteFinal cart
njmQqZ7WFcsKLw6Kwu7JJZRBI Bakeoff 2026-07-24NOAgent failed to read back order and escalated to a human employee for assistance.1x Classic Chicken Sandwich Large Combo (Cajun Fries, L
HukJQEqbbkSxkGHP3bRmQPSimulation Run 2026-07-20 22:35 (scratch)NOThe agent escalated to a human when the customer requested order read-back.1x Classic Chicken Sandwich Large Combo (Cajun Fries, M
6hSNXwKKgVz9MnVp4Zf8UURBI Bakeoff 2026-07-24no transcript—
4oEbLrnwemCqJPoYvq4yz9Simulation Run 2026-08-16 21:10no transcript—
9gyAMK7YqkB37hYzTKKa6HSimulation Run 2026-08-16 21:10no transcript—
9uS4XBqs5WFJjkdKsi6MZtSimulation Run 2026-08-16 21:10no transcript—
Syv2Lp7NXtRiC2G8uVKP7vSimulation Run 2026-08-16 21:10no transcript—
YCkd8QEC7WNh7w9aYGxebZSimulation Run 2026-08-16 21:10no transcript—
Z52GuM9up4ybUQqDW5cRM5Simulation Run 2026-08-16 21:10no transcript—
ZWE2ehWGkBbDpXVeBwjMP7Simulation Run 2026-08-16 21:10no transcript—
anrfJY9CiPUWeSeUsq8A3FSimulation Run 2026-08-16 21:10no transcript—
hC4BGNfiHDst5AgWTDrxc9Simulation Run 2026-08-16 21:10no transcript—
iJPq3Y6bfYkg2DdAtwbRmBSimulation Run 2026-08-16 21:10no transcript—
5KCuksaBVXmKBYhuBccYuMSimulation Run 2026-08-17 01:07UNKNOWNThe transcript does not include the agent directing the customer to pull forward or complete the order without escalation.1x Classic Chicken Sandwich
6hug2pBbuP2HqzWbWrByKfTest new prompt (Aug)no transcript—
6zPMkAJK8ZCwtF9Ac2PBWvTest new prompt (Aug)no transcript—
SxBhMYZPx3yBdbaRinDPaoTest new prompt (Aug)no transcript—
YCxe7uUmPY6Jybq6TEeMP5Test new prompt (Aug)no transcript—
i6RYuXvmUJonWXXwMsmhPsTest new prompt (Aug)no transcript—
jCjf2tYhL3cqC9VtxjTBLvRBI Bakeoff 2026-07-23YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Classic Chicken Sandwich Regular Combo (Cajun Fries,
ZcSCeTwHoYZhGFM5xDpwyjRBI Bakeoff 2026-07-24 Pass 5YESAgent fully captured the order and directed the customer to pull forward without escalation.1x Classic Chicken Sandwich
ftQKkRqnfbzsGCEWvnZkWVRBI BakeoffYESThe agent fully captured the customer's order and directed them to pull forward without escalation.1x Classic Chicken Sandwich
ATTjtj3EuiTTEQNRNsnLNnRBI BakeoffYESAgent captured the full order and instructed the customer to pull forward without escalating or handing off.1x Classic Chicken Sandwich Large Combo (Cajun Fries, M
PPjjv3gyKLvJgqMXNGQPcgSimulation Run 2026-08-16 23:02YESAgent took full order, confirmed it, then directed customer to pull forward without escalation.1x Classic Chicken Sandwich
b9hBLsEzWts4Yh4j69YmL4Simulation Run 2026-08-17 23:31YESThe agent took the full order and directed the customer to pull forward, without escalating to a human.1x Classic Chicken Sandwich

UAT #36

6/8 passed (75%)View in Coval →

I'll take the 3-piece chicken combo, spicy, add a pie - make it the strawberry - and a premium strawberry lemonade.

SimulationBatchVerdictJudge noteFinal cart
Gvo8rCud6DoLTtLkpG7hVtRBI Bakeoff 2026-07-24 Pass 5NOAgent escalated to a human team member instead of confirming the full order and directing customer forward.1x 3 Piece Signature Chicken Regular Combo (Spicy, Froz
TKDtcEnLqjtqRAFtX8VtPFRBI Bakeoff 2026-07-24NOThe agent did not read back the full order and deferred to a human employee for verification multiple times.1x 3 Piece Signature Chicken Regular Combo (Spicy, Chil
HsHzQktm9SztgJXTqZfpruRBI Bakeoff 2026-07-24no transcript—
3PP4DbUoQhUfgJyfwwS8URSimulation Run 2026-08-16 21:10no transcript—
3Pgzo7bCEKnWjJjw6UuLpYSimulation Run 2026-08-16 21:10no transcript—
6ccoCr6usxSbYcTU7Fo9SySimulation Run 2026-08-16 21:10no transcript—
JDta2tgNa3ChUY9PY3vfdZSimulation Run 2026-08-16 21:10no transcript—
KJdD5R2jojHfeJfLCEprM6Simulation Run 2026-08-16 21:10no transcript—
VkHYrroPcjQVSBhWCLFGcuSimulation Run 2026-08-16 21:10no transcript—
bakD6x7fe6KcMoJowJYBVASimulation Run 2026-08-16 21:10no transcript—
fS7g33j4eU4x7aMAzJCtoSSimulation Run 2026-08-16 21:10no transcript—
g8opK7n7xFLbXCC5m5dhPKSimulation Run 2026-08-16 21:10no transcript—
kPPC8XQoLcStx8Z7NmmhMCSimulation Run 2026-08-16 21:10no transcript—
4Sgc6Pu87cLKhWzmpjVpbJSimulation Run 2026-08-17 01:07UNKNOWNTranscript records only customer's order; no indication of agent taking order or directing customer forward.empty
68SAWb9yipxMS22b3f2ws4Test new prompt (Aug)no transcript—
DqWfkVg9BWMZjavQfLLVqwTest new prompt (Aug)no transcript—
RTDAMZZay7asWJqskfKEdnTest new prompt (Aug)no transcript—
RYnR42WoMQpfsaM5uBvBuDTest new prompt (Aug)no transcript—
cuC6jykzwZk76BTUE9JTekTest new prompt (Aug)no transcript—
iwEuVDuV7U7Lkeeue3vw3jRBI Bakeoff 2026-07-23YESAgent captured full order details and directed customer to pull forward without escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Froz
kNKkNjvRkM6p6N5Jng8z42RBI BakeoffYESAgent took full order, confirmed it, and directed customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Frozen
LmoBEJSNRrLRVfwXyz9hHJRBI BakeoffYESThe agent captured the full order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Chille
RsTg2Yd3PBCcgGqBx2BowNSimulation Run 2026-07-20 22:35 (scratch)YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Froz
LKanw6hW7m2TSUbVXKwLEoSimulation Run 2026-08-16 23:02YESThe agent took the full order, confirmed accuracy, and directed the customer to pull forward without escalation.1x Strawberry Cream Cheese Pie; 1x 3 Piece Signature Ch
kcvgvABQqTLnCwF6rFTdAiSimulation Run 2026-08-17 23:31YESThe agent took the full order including details and directed the customer to pull forward without escalation.1x Strawberry Cream Cheese Pie; 1x 3 Piece Signature Ch

Combo offer after signature chicken

12/15 passed (80%)View in Coval →

Order three pieces of spicy signature chicken by themselves. Do not ask for a combo. If the agent offers to make it a combo, accept and choose mashed potatoes and a Coke.

SimulationBatchVerdictJudge noteFinal cart
AjXYDUMMFsomxtPpVUciRASimulation Run 2026-07-20 22:35 (scratch)NOThe agent escalated to a human by stating an employee will assist the customer, not completing the order alone.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
FMGRr65XWLtreWbifwUhRoSimulation Run 2026-08-16 21:10NOThe agent did not direct the customer to pull forward or the next window; no completion of order or direction given.1x 3 Piece Signature Chicken (Spicy)
YuNoXdLUnpiRZfG3X7caxGSimulation Run 2026-08-16 21:10NOAgent failed to read back the order and suggested employee help, indicating escalation to a human.1x 3 Piece Signature Chicken (Spicy)
NLBrvvjnF8Y4kzGQA9L9tYRBI Bakeoff 2026-07-24no transcript—
6QbKRQJ5E6WTGZR9jre5W7Simulation Run 2026-08-16 21:10no transcript—
HUxTLWUMCwpkDQ2xGQykUsSimulation Run 2026-08-16 21:10UNKNOWNThe transcript lacks any indication the agent directed the customer to pull forward or the next window.1x 3 Piece Signature Chicken (Spicy)
ZPSdGCdB8BDAzwznFSbh3HSimulation Run 2026-08-16 21:10no transcript—
eDDiFhqnPzbp9TZzrKzruPSimulation Run 2026-08-16 21:10UNKNOWNTranscript shows order request but no completion or direction to pull forward.1x 3 Piece Signature Chicken (Spicy)
7T2Jqp8w9CeMcDn7NvPVYXTest new prompt (Aug)no transcript—
Vwy4pynTi67ypcgohc5MPETest new prompt (Aug)no transcript—
bzqRHWBNH4qj8DK49xjD52Test new prompt (Aug)no transcript—
mfYhvpBhbXWXQozPXuR9CtTest new prompt (Aug)no transcript—
oRMApRtJ8Rs5SU6NQtzZGdTest new prompt (Aug)no transcript—
JrFhk8w8KH9Av4wRw8wNh5RBI Bakeoff 2026-07-23YESAgent fully captured order and directed customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
cHuBpeq7PrxYyJbmofwwRWRBI Bakeoff 2026-07-24 Pass 5YESThe agent captured the complete order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
BTqpCuTCsSEsE58stbi3K9RBI Bakeoff 2026-07-24YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
ZtQ3rMbi44ADLWYpyQLh92RBI BakeoffYESAgent fully captured the order and directed customer to pull forward without escalating.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
7JQ5H95h3vPo9c53DUDi3JRBI BakeoffYESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
KYgDrWB5W2omtdMcXhh4wVSimulation Run 2026-08-16 21:10YESThe agent captured the full order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken (Spicy)
NUeJEy8gNXLfRRtAeXyc6aSimulation Run 2026-08-16 21:10YESAgent took full order, confirmed all items, and sent customer forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
Q5QEVWgHKnTXepiDQJue5fSimulation Run 2026-08-16 21:10YESAgent took the full order including sides and drink, confirmed it, and directed customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
YfYhbFjdgo3sT2PEjyyizxSimulation Run 2026-08-16 21:10YESAgent fully captured the order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
kNNSK8fxt8vDGzuWmwT4ekSimulation Run 2026-08-16 23:02YESThe agent fully captured the order, confirmed it with the customer, and directed them to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed
J9qAtCygx9yNSYXXN58qLSSimulation Run 2026-08-17 01:07YESThe agent fully captured the order and directed the customer to pull forward without escalating.1x 3 Piece Signature Chicken (Spicy)
LXCLmtqcKazSsJfxw7Uq9WSimulation Run 2026-08-17 23:31YESThe agent captured the full order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Large Combo (Spicy, Mashed

Alternative side offering

7/8 passed (88%)View in Coval →

I'd like the 3-piece signature chicken combo, spicy. I don't want fries. What other sides do you have?

SimulationBatchVerdictJudge noteFinal cart
8CdBxbi3ZDZuMQP6Hn2vRFSimulation Run 2026-07-20 22:35 (scratch)NOThe agent escalated to an employee without directing the customer to pull forward or the next window.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
6Nen9zEAoqAnvFQL7Pr2J9RBI Bakeoff 2026-07-24no transcript—
AfK6JK8QbNAaFu4XAfiPydSimulation Run 2026-08-16 21:10no transcript—
BPoG6SErJpKRtctjjPWBDgSimulation Run 2026-08-16 21:10no transcript—
FcpDLQNbxuAtxg3UVHX9GkSimulation Run 2026-08-16 21:10no transcript—
LhZjv8Mm9wYaVqChNiZvX9Simulation Run 2026-08-16 21:10no transcript—
Ujd75hG984QTBQSB8fHPQmSimulation Run 2026-08-16 21:10no transcript—
Vx6UmAhiJeVLf4o7ncPERDSimulation Run 2026-08-16 21:10no transcript—
cucDsrSgEHHBW9JYvUCigVSimulation Run 2026-08-16 21:10no transcript—
eDK7mFt4JMxLxpyhjXFWDMSimulation Run 2026-08-16 21:10no transcript—
mfMXdX9pP4dQq2Tu787MjKSimulation Run 2026-08-16 21:10no transcript—
njjh9bMWmfizHoQLhyzhFNSimulation Run 2026-08-16 21:10no transcript—
9ybUAWEtWs3NSKifvK4NddSimulation Run 2026-08-17 01:07UNKNOWNThe transcript does not show the agent taking the order or directing the customer forward.empty
93t8og6KrBYed48CucMCkeTest new prompt (Aug)no transcript—
XUg3Ysw32DgbsDoDGiHnkpTest new prompt (Aug)no transcript—
b96gnAtjPzmT7fgFSsRDH4Test new prompt (Aug)no transcript—
eAj94dknq9YyqnFrv4YcSATest new prompt (Aug)no transcript—
kPFZgAt965kzHLwdAiBQwHTest new prompt (Aug)no transcript—
etCLMJgMNaq3bhpqW8CV9LRBI Bakeoff 2026-07-23YESThe agent fully captured the order and directed the customer to pull forward without escalating or handing off.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
JHBNFZiNdiszaRwpomgvoeRBI Bakeoff 2026-07-24 Pass 5YESAgent fully captured the order and directed customer to pull forward without escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
K7FotYbpg5kpZk4ckQfSNyRBI Bakeoff 2026-07-24YESThe agent fully captured the order and directed the customer to pull forward without escalating.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
Q7LyZZ72wdPpjCLpuBGYuiRBI BakeoffYESAgent fully captured order and directly told customer to pull forward without escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
jWvDLHWXUsMWYmBfJar6DFRBI BakeoffYESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
XaCtGudYyg8uwqaQAocyuzSimulation Run 2026-08-16 23:02YESThe agent fully captured the order including modifications and directed the customer to pull forward without escalating.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
GTDQJEyzSpTnv3A55ZbtSSSimulation Run 2026-08-17 23:31YESThe agent fully captured the order, confirmed it, and directed the customer to pull forward without any escalation.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash

UAT #42

7/8 passed (88%)View in Coval →

Can I get the Ghost Pepper Sandwich combo? Actually, change that to the regular chicken sandwich. And make it spicy.

SimulationBatchVerdictJudge noteFinal cart
duJRLxY6i7YYjYkRNbUPDQRBI BakeoffNOThe agent escalated to a human team member instead of completing the order and sending the customer forward.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
8jPgoWyFgFgkWg5RwnsbVCRBI Bakeoff 2026-07-24no transcript—
DiaVkiLie7AutEZccjnK34Simulation Run 2026-08-16 21:10no transcript—
G8GGsxYDqC79ZF9pzEMxbXSimulation Run 2026-08-16 21:10no transcript—
JZcxeNfChi6j7dqh4zEXDHSimulation Run 2026-08-16 21:10no transcript—
L82TM8w8B5Haegnv6L5eEVSimulation Run 2026-08-16 21:10UNKNOWNThe transcript does not show the agent confirming the full order or directing the customer to pull forward or the next window.empty
MLg5fGU6P79qcsdqSUeRtnSimulation Run 2026-08-16 21:10no transcript—
RNFY6ziLiwp4dqRX4DCikZSimulation Run 2026-08-16 21:10no transcript—
UBninHbki4FsdVwRCNXQURSimulation Run 2026-08-16 21:10no transcript—
UQ8h3BiVHPNzpa4nZeRxDaSimulation Run 2026-08-16 21:10UNKNOWNThe transcript does not show the agent confirming the full order or directing the customer to pull forward or the next window.empty
eGgsLkK8qYDLfRcfwdPAmwSimulation Run 2026-08-16 21:10no transcript—
gk22nmmuTnt6CqXpJ9ym4ASimulation Run 2026-08-16 21:10no transcript—
NYMoL5RxWYLZyRCD4JAzKHSimulation Run 2026-08-17 01:07UNKNOWNTranscript does not show agent directing customer to pull forward or next window after taking order.1x Spicy Chicken Sandwich Regular Combo
8RHJvGhPXVNAiAENDSP6LNTest new prompt (Aug)no transcript—
EzPzPRYaJGJz4rh9rsaG3UTest new prompt (Aug)no transcript—
aPbwq4uXi22ufB5LPZo4ArTest new prompt (Aug)no transcript—
drLxycR9UKqJgUZUHLk5szTest new prompt (Aug)no transcript—
geRbswEXBXXiDPMPgDFbkpTest new prompt (Aug)no transcript—
N6CLvgSyUBvcJSkKCiZdjRRBI Bakeoff 2026-07-23YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Large Combo (Mashed Potatoes
XYvT9MrQnUbde3onRAfqE5RBI Bakeoff 2026-07-24 Pass 5YESAgent fully captured the order and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
bD7hBTNMBLhzjtjAzbPdn9RBI Bakeoff 2026-07-24YESThe agent took the full order and clearly told the customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
HuLEW9uUmZpnAVLzf7wbj4RBI BakeoffYESThe agent fully captured the order, confirmed it with the customer, and directed them to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
nFen2ZmsNLhH8ssFvkd82nSimulation Run 2026-07-20 22:35 (scratch)YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
hq7C8we95NxiiRDjqSUoF8Simulation Run 2026-08-16 23:02YESThe agent fully captured the order and directed the customer to pull forward without escalating.1x Spicy Chicken Sandwich Regular Combo (Cajun Fries, C
mw2SqX8fKz9kTupciZAyyYSimulation Run 2026-08-17 23:31YESAgent captured the full order including modifications and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Mas

UAT #69

7/8 passed (88%)View in Coval →

Do you have nuggets?

SimulationBatchVerdictJudge noteFinal cart
EsFDbbGBBZYkEk7n3sgfuqRBI BakeoffNOAgent escalated to a human employee instead of completing the order and directing the customer forward.1x Kids Tender Meal (Buttermilk Ranch)
XtbtYddcLBzKe8wbCww3P7RBI Bakeoff 2026-07-24no transcript—
44Vam9wWvvzjdjymAuMD5QSimulation Run 2026-08-16 21:10no transcript—
6seyRCLFHRvwqJXzvEDAVYSimulation Run 2026-08-16 21:10no transcript—
D5mScj6GrUkb4G5h8x4N4fSimulation Run 2026-08-16 21:10no transcript—
Ekr4XAUrzZCQb9Bmzygep6Simulation Run 2026-08-16 21:10no transcript—
LHAfvR3fUhWQ5StBFZaFh8Simulation Run 2026-08-16 21:10no transcript—
UXvQ8UNQRzTqLLpkthFNq9Simulation Run 2026-08-16 21:10no transcript—
WZF7KFB4L7cG56LeX2WSYLSimulation Run 2026-08-16 21:10no transcript—
iXsdsiByUaCaofy84unEC2Simulation Run 2026-08-16 21:10no transcript—
kWDjkX5sLhebV5draGXVBpSimulation Run 2026-08-16 21:10no transcript—
mVogxmJRdfC95EPVfzPUTfSimulation Run 2026-08-16 21:10no transcript—
ZobydacNshzrTqtDVLq6Q8Simulation Run 2026-08-17 01:07UNKNOWNThe transcript does not contain any interaction about taking or directing the customer's order.empty
BmBtwnDa4LuVmgLAryHnVwTest new prompt (Aug)no transcript—
GnK5tEQCvtSHCXWE3QMb2WTest new prompt (Aug)no transcript—
HArzCParVTQ2Hwg6Tfgz9kTest new prompt (Aug)no transcript—
PsX5sa8NBujzEehzHLR9EBTest new prompt (Aug)no transcript—
ZSvbBkG8vc4xBMAqjAeFQLTest new prompt (Aug)no transcript—
CSa66eud6bzyCtzGNwSU7SRBI Bakeoff 2026-07-23YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Large Combo (Blackened Ranch
4p8zjB2qV8WuBQSNXGBPRWRBI Bakeoff 2026-07-24 Pass 5YESAgent captured full order details and directed customer to pull forward without escalation.1x 5 Piece Chicken Tenders Large Combo (Cajun Fries, Wi
bcGT5aZvMjmQgUNP7aMwNHRBI Bakeoff 2026-07-24YESThe agent captured the full order including modifications and directed the customer to pull forward without escalating.1x 5 Piece Chicken Tenders Large Combo (Cajun Fries, Ma
XELGy6ySsZMuEKLF3aviCoRBI BakeoffYESThe agent fully captured the customer's order and directed them to pull forward without escalating.1x 5 Piece Chicken Tenders Large Combo (Blackened Ranch
ekBTGz6xmYSkPC6Jd94XsKSimulation Run 2026-07-20 22:35 (scratch)YESAgent captured the full order and instructed the customer to pull forward without escalating.1x 3 Piece Chicken Tenders Regular Combo (Classic, Caju
382fTnFbeUpyr8LF2rK4aTSimulation Run 2026-08-16 23:02YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Large Combo (Homestyle Mac a
WMRo9NvhBHrfVsUwNonPS9Simulation Run 2026-08-17 23:31YESThe agent fully captured the customer's order and directed them to pull forward without escalating or handing off.1x 5 Piece Chicken Tenders Large Combo (Blackened, Caju

UAT #1

8/9 passed (89%)View in Coval →

One Ghost Pepper Sandwich, no combo.

SimulationBatchVerdictJudge noteFinal cart
nyKeMy3c3tRRqxnmBCkFsqSimulation Run 2026-08-17 01:07NOAgent failed to direct customer to pull forward or next window after taking order, and order wasn't fully finalized.1x Spicy Chicken Sandwich
9rZXsbyFCu3CEhp5EYdg6gRBI Bakeoff 2026-07-24no transcript—
3as2dUBpDSxLwAx9LQRdwjSimulation Run 2026-08-16 21:10no transcript—
3bZb73mNGLht3JK5VSaX9KSimulation Run 2026-08-16 21:10no transcript—
6xG2rPbTGpnkvrgW6CbTrgSimulation Run 2026-08-16 21:10no transcript—
EwUEcYSaHvuG7JmtZkozPaSimulation Run 2026-08-16 21:10no transcript—
JJCNShskiNT2EUd3Z8RNqvSimulation Run 2026-08-16 21:10no transcript—
L9RWcLxWfdE28iMwEddG4pSimulation Run 2026-08-16 21:10no transcript—
LNSTWN2HoLwTDeQs6Ay2fJSimulation Run 2026-08-16 21:10no transcript—
LV7MhJBQBTbtzz9JhTNp9BSimulation Run 2026-08-16 21:10no transcript—
THwL3XsdBaiNnLPucnzVgfSimulation Run 2026-08-16 21:10no transcript—
aMqXz7rnrsxkzUE7w6DH9jSimulation Run 2026-08-16 21:10no transcript—
7qVZqk4LZc9R6EEWgAPCXVTest new prompt (Aug)no transcript—
8cPokrBr3gvCjW6j2pPwraTest new prompt (Aug)no transcript—
G72DQqBgfHfEdyWEheek7iTest new prompt (Aug)no transcript—
WDMhVQKkjAdMngJU6esz9nTest new prompt (Aug)no transcript—
ae9brQq9sWnLtGfr8cAtguTest new prompt (Aug)no transcript—
f5AoKWT3Xsu492SNjtDaa5RBI Bakeoff 2026-07-23YESThe agent captured the full order and directed the customer to pull forward with no escalation or handoff.1x Spicy Chicken Sandwich
6WNc74sgwcUudYhgFAo5BZRBI Bakeoff 2026-07-24 Pass 5YESThe agent took the full order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich
7US5UtBNubTuKxP9M8eEvRRBI Bakeoff 2026-07-24YESThe agent fully captured the order and directed the customer to pull forward without any human escalation.1x Spicy Chicken Sandwich
GVwFnNWqtSoWUk4KiyEhb4RBI BakeoffYESAgent took full order, confirmed it, and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich
cCJWSjbpDj92opMF3uAkdzRBI BakeoffYESAgent took full order of spicy chicken sandwich and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich
ajuwNDBzsF4v3BHyJuRC3wSimulation Run 2026-07-20 22:35 (scratch)YESThe agent fully captured the customer's order and directed them to pull forward without escalating.1x Spicy Chicken Sandwich
ghGLV6mMBGYxxb3dPwfguDSimulation Run 2026-08-16 23:02YESThe agent fully captured the order and instructed the customer to pull forward without escalation.1x Spicy Chicken Sandwich
f9MFgTJbzUaPF3DGP4i5WCSimulation Run 2026-08-17 23:31YESThe agent took the full order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich

UAT #10

8/9 passed (89%)View in Coval →

Can I get the ghost pepper sandwich combo, swap the fries for red beans & rice, and a Dr Pepper?

SimulationBatchVerdictJudge noteFinal cart
PAr7nXjhCLSW4a6jrUDVEXSimulation Run 2026-08-17 01:07NOAgent did not confirm or complete the order or direct the customer to pull forward; no order completion observed.empty
cwaibRJC3SLk28hzWoibbeRBI Bakeoff 2026-07-24no transcript—
59a93oSq5KNxfgvPoMYZdkSimulation Run 2026-08-16 21:10no transcript—
6P9TAYyFiD9LJFcQqE3jrdSimulation Run 2026-08-16 21:10no transcript—
HxqDvBwEzfQG96xxR5G8hbSimulation Run 2026-08-16 21:10no transcript—
KNupqvwiRRDAu4eVvuRHJXSimulation Run 2026-08-16 21:10no transcript—
SEQ6eqTuokCW7mk2FxNSPWSimulation Run 2026-08-16 21:10no transcript—
WvyqTmwNac4WteV4XkouYiSimulation Run 2026-08-16 21:10no transcript—
YDDQrmCaRKjhYCgqVeL2uGSimulation Run 2026-08-16 21:10no transcript—
aqe2CYFYTrzVQ2ibsJiWzsSimulation Run 2026-08-16 21:10no transcript—
hUwJLc9VewjXtmbhVDt2NFSimulation Run 2026-08-16 21:10no transcript—
jNQSTPu8WG2LUyacJaMYU8Simulation Run 2026-08-16 21:10no transcript—
S2kCEq9aopACSJkGPknUjKTest new prompt (Aug)no transcript—
ZJUUM7DBocQP7om82YSr4iTest new prompt (Aug)no transcript—
gyfUYiKamFnx73oz5vHY7ATest new prompt (Aug)no transcript—
hEA7PoTi5hQYM5VWvHvDFCTest new prompt (Aug)no transcript—
jwv2EiCAGRSzWTjDxCS9JQTest new prompt (Aug)no transcript—
dNUEN3jB7SBu3PZjLaAyyLRBI Bakeoff 2026-07-23YESOrder was fully captured with sides and drink, and customer was directed to pull forward without escalation.empty
ciffMRzRzBdKzhGBkTLSL2RBI Bakeoff 2026-07-24 Pass 5YESThe agent fully captured the order with modifications and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Red Beans and
7XyEW37phzT8wnhn2okEJMRBI Bakeoff 2026-07-24YESAgent captured full order with modifications and asked customer to pull forward without escalating.1x Spicy Chicken Sandwich Regular Combo (Red Beans and
2p7khitxWyfVdEWyXXkevvRBI BakeoffYESThe agent fully captured the order including sandwich, sides, and drink, then directed customer to pull forward without escalation.1x Spicy Chicken Sandwich Large Combo (Red Beans and Ri
Ej5yJ9T39KETceTnQZKhyARBI BakeoffYESAgent fully captured the order including modifications and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Red Beans and
ZmdgBKsAMrNLTAHY4WmeTgSimulation Run 2026-07-20 22:35 (scratch)YESThe agent captured the full order including modifications and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Red Beans and
Nhsw4nLwy2ACjjUo8Ep3UeSimulation Run 2026-08-16 23:02YESAgent took full order, confirmed it, and directed customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Red Beans and
5FFYi5baehFca2qdApME6bSimulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Regular Combo (Red Beans and

UAT #11

8/9 passed (89%)View in Coval →

I'll do the chicken sandwich by itself and a side of fries.

SimulationBatchVerdictJudge noteFinal cart
cVENNXs6m7mWv8apN3x6QdSimulation Run 2026-08-17 01:07NOThe agent did not take the full order nor direct the customer to pull forward or the next window.empty
MeaSJHjZt6sv253oMjwLxmRBI Bakeoff 2026-07-24no transcript—
6hdoCPquJsL36LiUkcDbgwSimulation Run 2026-08-16 21:10no transcript—
D9FtrP5kHSXhqmA9w9SA73Simulation Run 2026-08-16 21:10no transcript—
ERUkoaRstVNKnvaAZ7Ts5BSimulation Run 2026-08-16 21:10no transcript—
FL3ixSkaX3g6BYgncDx6XqSimulation Run 2026-08-16 21:10no transcript—
KUV8Q4b5jKkSRBCXSf8QMjSimulation Run 2026-08-16 21:10no transcript—
Nq3KdYomJt2Qf37LSgnYcZSimulation Run 2026-08-16 21:10no transcript—
PfN6xtQeduztmRpHBZN4mhSimulation Run 2026-08-16 21:10no transcript—
dtonDSfqcyxFZekDB6SjnQSimulation Run 2026-08-16 21:10no transcript—
gaSBiT8NZsMmFLQuAsoi93Simulation Run 2026-08-16 21:10no transcript—
jSvzXJEG22XPaHEZxHEtVrSimulation Run 2026-08-16 21:10no transcript—
3azSNHg9mofWGjs9QAmGD2Test new prompt (Aug)no transcript—
G6wpWZL9M4DnhLhMZj78tXTest new prompt (Aug)no transcript—
XXhKL6q43aqnDmihu4HSHZTest new prompt (Aug)no transcript—
YRbhYStkUB4CptVws36A4mTest new prompt (Aug)no transcript—
dobtdHtUusWma34DYmeE2LTest new prompt (Aug)no transcript—
FgC87aaqGSii3yNCWCZWn4RBI Bakeoff 2026-07-23YESAgent fully captured the order and directed customer to pull forward without escalation.1x Classic Chicken Sandwich; 1x Cajun Fries
ZdiQaU64Tumg7XrxmTPTG7RBI Bakeoff 2026-07-24 Pass 5YESAgent captured full order and directed customer to pull forward without escalation.1x Classic Chicken Sandwich; 1x Cajun Fries
YrZ9WsuKjd6ncDateUKTfuRBI Bakeoff 2026-07-24YESAgent captured full order and directed customer to pull forward without escalation.1x Classic Chicken Sandwich; 1x Cajun Fries
kCeMGUNbDk2h5TGAnuFCiwRBI BakeoffYESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Classic Chicken Sandwich; 1x Cajun Fries
PPyjSTyQTRzBf49MVksLt4RBI BakeoffYESAgent fully captured order and directed customer to pull forward without escalating to a human.1x Classic Chicken Sandwich; 1x Cajun Fries
M8BdUEGntkiWNvCofEnh4tSimulation Run 2026-07-20 22:35 (scratch)YESThe agent fully captured the order and directed the customer to pull forward without escalating.1x Classic Chicken Sandwich; 1x Cajun Fries
PcTh8tvTQgCtB9AX66JPE9Simulation Run 2026-08-16 23:02YESAgent fully captured the order and directed the customer to pull forward without escalation.1x Classic Chicken Sandwich; 1x Cajun Fries
UiHeqTECCDKbfyF8iSpeJ2Simulation Run 2026-08-17 23:31YESThe agent took the full order and directed the customer to pull forward without escalating or bailing.1x Classic Chicken Sandwich; 1x Cajun Fries

UAT #8

8/9 passed (89%)View in Coval →

I'd like the 5-piece tenders combo, blackened and spicy, with cole slaw and a Coke.

SimulationBatchVerdictJudge noteFinal cart
YwGxspS7cYaWpqGG5eZSkGSimulation Run 2026-08-17 01:07NOThe transcript does not show the agent directing the customer to pull forward or the next window after order completion.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Sp
hctaB2TxF3MtBmJGnPteVaRBI Bakeoff 2026-07-24no transcript—
3L87kh85hZ9vwMw8roLmyUSimulation Run 2026-08-16 21:10no transcript—
3m84J8VQSWwmGBXAhSQPPtSimulation Run 2026-08-16 21:10no transcript—
5SXV5Vn4jcDnL4Jz22j3m6Simulation Run 2026-08-16 21:10no transcript—
9P3f8tN99kS3iGaWQ5mpkbSimulation Run 2026-08-16 21:10no transcript—
MQqixCuNEJXMzubTth3L6ASimulation Run 2026-08-16 21:10no transcript—
TadyGkTFKCYJBt5ztF4xD4Simulation Run 2026-08-16 21:10no transcript—
UwwNgveTfhuCZ92RkSfUNqSimulation Run 2026-08-16 21:10no transcript—
V9rVS95MyF5BNWG3oNATFCSimulation Run 2026-08-16 21:10no transcript—
Z6YJU65eHiWocjdphmFhaTSimulation Run 2026-08-16 21:10no transcript—
nLzovCGALzikVT6fc2aUksSimulation Run 2026-08-16 21:10no transcript—
KRrt6fg3zQYv4Eu9RS8b5rTest new prompt (Aug)no transcript—
PVW39TwMFa5r2yiPzicTQmTest new prompt (Aug)no transcript—
bNCFRrvgin2drckB6LgqpfTest new prompt (Aug)no transcript—
kSK7QqRZzkeYU6ymgUEjHUTest new prompt (Aug)no transcript—
miiA7KgrQuwgiuyuqdrhunTest new prompt (Aug)no transcript—
6viQvJL6ZB4YQWvpqDMv2CRBI Bakeoff 2026-07-23YESThe agent captured the full order, confirmed it, and instructed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
6kWmh6wUMDRYzDLEbJGU5fRBI Bakeoff 2026-07-24 Pass 5YESThe agent fully captured the customer's order and instructed them to pull forward without any escalation.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
gUrM5pLgWq7u4pUdXfLww7RBI Bakeoff 2026-07-24YESThe agent fully captured the order, confirmed its accuracy, and directed the customer to pull forward without escalating.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
N9LMABRYJx3QYjEKbgmQ97RBI BakeoffYESThe agent took the full order, confirmed it, and directed the customer to pull forward without escalating.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
9bNAxJwjtYLCMD5Ae4bix9RBI BakeoffYESThe agent fully captured the order, confirmed it, and directed the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
BuNFyG9mt9jyoFbMMJyTmoSimulation Run 2026-07-20 22:35 (scratch)YESThe agent fully captured the order, confirmed its accuracy, and directed the customer to pull forward without escalating.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Co
Q2nRJPLsi8ymqLcyDg4HexSimulation Run 2026-08-16 23:02YESAgent fully captured the order and told the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Sp
nEGUqaXbUV7odSwqBVByGLSimulation Run 2026-08-17 23:31YESAgent fully captured the order and told the customer to pull forward without escalation.1x 5 Piece Chicken Tenders Regular Combo (Blackened, Sp

Mid-order quantity change

9/10 passed (90%)View in Coval →

Start by ordering the 3-piece signature chicken combo, spicy, with mashed potatoes and a Coke. Before the order is finalized, change it to the 4-piece combo.

SimulationBatchVerdictJudge noteFinal cart
n4gmoKojMfqKQ5d4iMFbwKSimulation Run 2026-08-16 21:10NOThe agent did not confirm the full order or direct the customer to pull forward or the next window.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
eofuuonX2y4uYkqHfoRGxxRBI Bakeoff 2026-07-24no transcript—
Faq36JKHqeWn3gT6MjXnZYSimulation Run 2026-08-16 21:10no transcript—
JSKWgy4YRJDBohJpPtSwLESimulation Run 2026-08-16 21:10no transcript—
KeE9e39QFoQwT6rNWtt3PnSimulation Run 2026-08-16 21:10no transcript—
PYsfqzYrEyAKiQS2pNM68FSimulation Run 2026-08-16 21:10no transcript—
VFWUGNhCyVUNRACEAShjdTSimulation Run 2026-08-16 21:10no transcript—
VbGUjpvoFcLGdhBWdEjVaxSimulation Run 2026-08-16 21:10no transcript—
Vm8F8DEs7B5pHwYa4ageknSimulation Run 2026-08-16 21:10no transcript—
jtKjcYWZyVhmT69ADXTzm5Simulation Run 2026-08-16 21:10no transcript—
nAqr4GGXfDorXu5x2w6jSFSimulation Run 2026-08-16 21:10UNKNOWNTranscript lacks agent confirmation of full order and directing customer to pull forward/next window.1x 3 Piece Signature Chicken Regular Combo (Spicy, Mash
LD9qYMuaH68BpPtsUfEtiUTest new prompt (Aug)no transcript—
NLtKk8T8HUbfNrG6uzXoF5Test new prompt (Aug)no transcript—
NfzWAgsfoQvtCWdn3hUUNTTest new prompt (Aug)no transcript—
QPK6CYJ7xfgb8KaugzLxxuTest new prompt (Aug)no transcript—
UB95WDfD9hkDojtLktoVVHTest new prompt (Aug)no transcript—
DcQo3e9Txqrrd7GdDmTogjRBI Bakeoff 2026-07-23YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
GTbJJ44fSKNSgRckBYauKDRBI Bakeoff 2026-07-24 Pass 5YESThe agent fully captured the modified order and directed the customer to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
BqmydQNFa2nkuWd7bQXt6eRBI Bakeoff 2026-07-24YESThe agent captured the full order and directed the customer to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
AFr3aJqF2RcVuX4mJPkRocRBI BakeoffYESAgent fully captured the four-piece combo order with modifications and told customer to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
W8KnRNTM9M3iTLUtQgQYtjRBI BakeoffYESThe agent took the full order, confirmed it with the customer, and directed them to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
C7GPnbvcj682ZQ4cdor7y5Simulation Run 2026-07-20 22:35 (scratch)YESAgent captured the full order and directed customer to pull forward without escalating or handing off.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
Liv9RgEygopRpcA3CLea33Simulation Run 2026-08-16 23:02YESThe agent fully captured the order and directed the customer to pull forward without escalating or handing off.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
QsG68g9sAcHdkbpSjFi6qcSimulation Run 2026-08-17 01:07YESAgent fully captured the customer's order and directed them to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash
DjJxtzxn9aQ2VvVyVCsoLsSimulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x 4 Piece Signature Chicken Regular Combo (Spicy, Mash

UAT #64

17/18 passed (94%)View in Coval →

Can I have a Cajun Rice please?

SimulationBatchVerdictJudge noteFinal cart
R47RU2VwngJLKkCDRb3FhbSimulation Run 2026-07-20 22:35 (scratch)NOAgent did not complete the order themselves and indicated that an employee could help with unavailable item.1x Red Beans & Rice
mSwqc33nbEEQdi5baAqBEyRBI Bakeoff 2026-07-24no transcript—
9FWWd7RgWKyTduWj4RCPeTSimulation Run 2026-08-16 21:10no transcript—
BM456MR9eDQw2DtS5UQda9Simulation Run 2026-08-16 21:10no transcript—
R4WpyPCjocr5ATqAi6LhcgSimulation Run 2026-08-16 21:10no transcript—
S2daJTaST5tmjyvG6FvsXxSimulation Run 2026-08-16 21:10no transcript—
SdLhKpwfYqxiPV3WocSsHzSimulation Run 2026-08-16 21:10no transcript—
WKsuyFzD2ohVvscAgfdJEESimulation Run 2026-08-16 21:10no transcript—
ZQRSPKb2bFhADn2JVWCE34Simulation Run 2026-08-16 21:10no transcript—
ZfdtqcfPAN8AFvuHFutC8gSimulation Run 2026-08-16 21:10no transcript—
iH6B4zSr38G69Az3rpkUF2Simulation Run 2026-08-16 21:10no transcript—
jAd7mewmyLqbpFehJF9LLqSimulation Run 2026-08-16 21:10no transcript—
fkf7qmyNzG7ZQxuaN35jzzSimulation Run 2026-08-17 01:07UNKNOWNThe transcript does not show any order being taken or customer directed to pull forward; conversation is incomplete.empty
6ir2X2y9mwTpC4B8URrKpVTest new prompt (Aug)no transcript—
8DFm9GB7J6wUAPTtGYYNj5Test new prompt (Aug)no transcript—
B23cbJyECdTXXuvxj4oNuqTest new prompt (Aug)no transcript—
DFCKgtBneFnPWzNFBF3K8rTest new prompt (Aug)no transcript—
MRFWvgS8kyVgg3xcft7r5nTest new prompt (Aug)no transcript—
2vWLVf22BBP87G7oQ2LFr7RBI Bakeoff 2026-07-23YESAgent fully captured order and instructed customer to pull forward without escalation.1x Red Beans & Rice
iETFc6cWBjVosumqdDtEL5RBI Bakeoff 2026-07-24 Pass 5YESThe agent captured the full order (Red Beans & Rice, Regular size) and directed the customer to pull forward without escalation.1x Red Beans & Rice
LuaHhVFRShMdAkuH3mXDKFRBI Bakeoff 2026-07-24YESThe agent fully captured the order and instructed the customer to pull forward without escalation.1x Red Beans & Rice
VW2CEmiE4HWvQz7muCkoaCRBI BakeoffYESThe agent fully captured the order for red beans and rice and directed the customer to pull forward without escalation.1x Red Beans & Rice
UP2ncKsbCAnHJNGv5P9LzBRBI BakeoffYESAgent took the full order of red beans and rice, confirmed it with the customer, and directed them to pull forward without escalation.1x Red Beans & Rice
6BYZ5i3iKFMNAePbebfTqWSimulation Run 2026-08-13 22:42YESAgent captured full order of large red beans and rice and directed customer to pull forward without escalation.1x Red Beans & Rice
7yiZ9kqeRwEBeUDB2izkaESimulation Run 2026-08-13 22:42YESAgent fully captured the order for red beans and rice and directed the customer to pull forward without escalation.1x Red Beans & Rice
8hsNr9pXLTy8s5PbzvHbqiSimulation Run 2026-08-13 22:42YESThe agent fully captured the order and directed the customer to pull forward with no escalation.1x Red Beans & Rice (Rice)
A4RijE6zvaXoC4apqRqZAUSimulation Run 2026-08-13 22:42YESThe agent fully captured the order and directed the customer to pull forward without escalation.1x Spicy Chicken Sandwich Large Combo (Cajun Fries, Red
StZLPaz6j9ymHZgUMk6McVSimulation Run 2026-08-13 22:42YESThe agent fully captured the order and directed the customer to pull forward without escalation or handoff.1x Red Beans & Rice
VPgbu7DMFdhc9D2ua8F4G5Simulation Run 2026-08-13 22:42YESAgent fully captured order and directed customer to pull forward without escalation.1x Red Beans & Rice (Rice)
Xbevdez8XDTAqqJzAxtitnSimulation Run 2026-08-13 22:42YESThe agent fully captured the order, confirmed accuracy, and directed the customer to pull forward without escalation.1x Red Beans & Rice; 1x 3 Piece Signature Chicken Large
bbuH6UDxtsPpQNzakzVAs8Simulation Run 2026-08-13 22:42YESThe agent fully captured the order and directed the customer to pull forward without escalation.empty
fWniQjDFKSxJqTQorGVhsFSimulation Run 2026-08-13 22:42YESThe agent fully captured the order and asked the customer to pull forward without escalation.1x Red Beans & Rice
jUmDNTJmDp6bThNxxjGz4sSimulation Run 2026-08-13 22:42YESThe agent fully captured the order for regular red beans and rice and directed the customer to pull forward without escalating.1x Red Beans & Rice (Rice)
EnKWkb8varFiRsGenmocTQSimulation Run 2026-08-16 23:02YESThe agent took the full order and directed the customer to pull forward without escalating to a human.1x Red Beans & Rice (Rice)
DAhEXmkfjUun8jKxEpr4j6Simulation Run 2026-08-17 23:31YESThe agent fully captured the order and directed the customer to pull forward without escalating.1x Red Beans & Rice (Rice)