Ghost Pepper UATs: silent substitution

← Back to the full Coval report. The Ghost Pepper Sandwich appears in three UAT scenarios but is not in the POS menu. This page breaks down what the agent actually did in every judged conversation, pulled 2026-08-12.

Findings

  1. The Ghost Pepper Sandwich is not in the POS menu. Across all 288 judged conversations the cart contained a Ghost Pepper item 0 times. Every completed order carted a substitute, usually Classic or Spicy Chicken Sandwich, and the choice is inconsistent: on UAT #10 the customer gets Classic about as often as Spicy.
  2. Headline completion (73.3% across the three UATs) is dragged by the screen artifact, not Ghost Pepper handling. Adjusted completion is 88.6% (UAT #1), 95.4% (UAT #10), 97.3% (UAT #42).
  3. Disclosure is the real gap: the agent said the item was unavailable in 163 of 288 conversations. Ordered alone (UAT #1) it disclosed 88% of the time; buried in a combo order with modifications (UAT #10) that drops to 67%.
  4. Failure patterns, worst first: false affirmation ("Got it. Available right now."), silent substitution straight into the large-combo upsell, and the flavor steer ("Classic or spicy?") that treats Ghost Pepper as if it were a flavor choice.
  5. Coval's completion judge does not test disclosure: 24 silent substitutions were judged Completion=YES. RBI's UAT review does test it, and treats silent substitution as a failure (the UAT #64 Cajun Rice precedent). Same fix class: disclose unavailability, then recommend an alternative.

UAT #1

“One Ghost Pepper Sandwich, no combo.”

Judged completion
81.2%
Adjusted (screen artifact removed)
88.6%
Disclosed not available
84/96
Ghost Pepper carted
0
Disclosed Stalled or no reply Substitution risk

Click a bar for every conversation in that bucket, with transcript links.

Every conversation that did not disclose

BehaviorSimulationCompletionAgent's first replyFinal cart
Silently substituted2VnSDr8hjGzG52bK7GbNusNOSure. Anything else?empty
Silently substitutedFoZpDqPjUDEEP52h3TdVT7NOSure. Anything else?empty
Silently substituted8xbxnWmcY2yZdzrLDgVGeoYESIs your order at the bottom of the screen on the right correct?empty
Silently substitutedBCYd2taWD9nPWtxYyGhcPqNOGot it. Anything else?1x Spicy Chicken Sandwich
Silently substitutedJ688aB9ZNaYtWMzZpH5qDNNOSure. Flavor sandwich would you like?empty
Silently substitutedVgtP3jki2aueVB5Ue7ZzqcNOSure. Anything else?1x Spicy Chicken Sandwich
Steered without disclosing3Rzv3tq9RzuQxxJ6nztNVNYESSure. Have classic spicy or honey BBQ sandwiches. Which flavor?1x Spicy Chicken Sandwich
Steered without disclosingGCuN9rZVbtycJ8DprTZAvsYESSure. What sandwich would you like instead? Classic, spicy, or honey BBQ?1x Spicy Chicken Sandwich
Steered without disclosingSagfnTweXUrwmZq8M5UEREYESSorry. We only have classic spicy honey BBQ or bacon and cheese sandwiches. Anything else?1x Honey BBQ Chicken Sandwich
Steered without disclosing3tKQRrLm8ybBTf9FXugAPCNOMaybe so. Which sandwich would you like instead?empty
Steered without disclosingh8zgnCK3dM97RusuBrWhdFNOSure. Classic chicken sandwich or spicy chicken sandwich?1x Spicy Chicken Sandwich
Steered without disclosing4STd2TwmSrDXXvQBLsUt69NOYes. Classic or spicy?1x Spicy Chicken Sandwich

UAT #10

“Can I get the ghost pepper sandwich combo, swap the fries for red beans & rice, and a Dr Pepper?”

Judged completion
64.6%
Adjusted (screen artifact removed)
95.4%
Disclosed not available
64/96
Ghost Pepper carted
0
Disclosed Stalled or no reply Substitution risk

Click a bar for every conversation in that bucket, with transcript links.

Every conversation that did not disclose

BehaviorSimulationCompletionAgent's first replyFinal cart
False affirmationf7ReLz34fP5dwU2fdX5WZwYESGot it. Available right now. Anything else?1x Spicy Chicken Sandwich Regular Combo (Red
False affirmationb5kG8xdo9GCeLqEqKiRMFGYESIt has a noticeable kick, not extreme heat, We have classic or spicy sandwiches today.1x Spicy Chicken Sandwich Large Combo (Red Be
Silently substituted6gv7FeZMXry3Qd2XyoGtznNOGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substituted7Ffci9hf27qgGvwiGmz7VQYESGot it. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substituted7UnjnrisqzeUZeij2GowkaNOGot it. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substitutedB3KjWAGSsFeqBYfH45HcdyYESSure. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substitutedDAs4btzGmYoEaM466kCcoYYESGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedPQaJv8Hn5QqhFCDbwHm9WgYESSure. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substitutedS2TimLJyCTBvu5erqWqiupYESSure. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Dr.
Silently substitutedWgD6fsG3rtiC2Dz4FUDhk7NOGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedJeUNBfrF4vFwFRQ7qAVk5sYESGot it. Would you like to upgrade to a large combo?1x 2 Piece Signature Chicken Regular Combo (R
Silently substitutedJp8PCimHbTfgUtuvi3vgLuYESGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedP8bSLwf2ySSbvmrpPUgfNeNOGot it. Would you like to upgrade to a large combo?empty
Silently substitutedQxFxkCiwVPb7kZJbgwzKCSYESGot it. Would you like to upgrade to a large combo?1x Red Beans & Rice; 1x Dr. Pepper
Silently substitutedSgK4m572F7oKUKcjcQgXULNOPerfect. You like to upgrade to a large combo?1x Spicy Chicken Sandwich Large Combo (Red Be
Silently substituteddNUEN3jB7SBu3PZjLaAyyLYESGot it. Would you like to upgrade to a large combo?empty
Silently substituted3y9pLtRVwbCnZM6j7DsRvrNOPerfect. Is your order at the bottom of the screen on the right correct?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedNNCBn4kfN7i7FRo55uXwV3YESGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedg3LXnXYUwinTWhmHGmTWEGYESGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substituted3SaGtVLsRP8kTrk5LSgoJxNOGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedAjVMUFH2HNuH39NERQGtxUYESSure. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substitutedLDXtaKxAjARmsm7m4BrnswNOGot it. Your order at the bottom of the screen on the right correct?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedYAQnxFtpHymC7aW6ozHXtyNOGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedHnwTGAbx74YVHyNkoqHebbYESGot it. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedLbcGGKn8wxTw4vEDNqFE9qYESGot it. Would you like to upgrade to a large combo?1x Spicy Chicken Sandwich Regular Combo (Red
Silently substitutedW5SDL4XfrRP3gskVPZ3djwNOPerfect. Is your order at the bottom of the screen on the right correct?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedA59KnhTFDEbHzysSPDNBhzYESSure. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Large Combo (Red
Silently substitutedKoYErXKTB6AsYmPcPdBGbCYESSure. Would you like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedbrGJb9Qr4ErBwDRy7CaJqWYESPerfect. You like to upgrade to a large combo?1x Classic Chicken Sandwich Regular Combo (Re
Silently substitutedca2aqB5u9Vd7xLB4NshAd5YESSure. Have red beans and rice. What side dish and drink?1x Spicy Chicken Sandwich Large Combo (Red Be
Stalled, never answered8uugsoSvaqMaAxo3ko98YaNOOne moment while I check.empty
Steered without disclosingDNbjiSaZxMdiQXoiPhM4XvNOSure. Have classic or spicy chicken sandwich combos. Which would you like?1x Spicy Chicken Sandwich Regular Combo (Red

UAT #42

“Can I get the Ghost Pepper Sandwich combo? Actually, change that to the regular chicken sandwich. And make it spicy.”

Judged completion
74.0%
Adjusted (screen artifact removed)
97.3%
Disclosed not available
15/96
Ghost Pepper carted
0
Disclosed Stalled or no reply Substitution risk

Click a bar for every conversation in that bucket, with transcript links.

No table here: the customer changes to a regular sandwich on their own, so non-disclosure is mostly benign in this scenario.