The Chatbot Told Her Shipping Was Free. It Was Not.
AI chatbots on storefronts invent shipping times, free-shipping thresholds, and delivery promises the store never made. The hallucination shows up in the support queue, not in the chatbot's own metrics.
A shopper asks the store's AI chatbot whether her order will arrive before Friday. The bot says yes, confidently, and cites a two-day delivery window that does not exist for her postcode. She orders. On Friday there is no parcel, and the support ticket that follows quotes the chatbot's own words back at the store. The bot's dashboard shows the conversation as resolved.
This is the failure mode of storefront chatbots that nobody measures. The bot is graded on deflection, on how many conversations never reach a human, while the cost lands in refunds, reships, and chargebacks when the answer it gave was invented. Shipping is the worst category, because the answer depends on postcode, weight, carrier, and cutoff time, and the bot has none of those in its prompt.
Why shipping answers go wrong first
Shipping is a lookup, not a conversation. The right answer needs the shopper's postcode, the items in the cart, the warehouse they ship from, and the carrier's cutoff, which on most stores is somewhere between 1pm and 3pm. A chatbot that answers from its training data instead of a live lookup will give the average answer, and the average answer is wrong for most postcodes.
Free-shipping thresholds are the second trap. The bot says shipping is free over 50 dollars, because that is what it read in an old help article, while the store raised the threshold to 75 dollars three months ago. The bot's knowledge is a snapshot, and nobody owns updating it when the policy changes.
What the support queue shows
The pattern in the tickets is consistent: the shopper quotes the bot, the agent checks the real policy, and the store pays. Sometimes it pays in a refunded shipping fee, sometimes in a reshipped order, sometimes in a lost customer who will not trust the store's own website again. The cost per bad answer is small, usually the price of expedited shipping, but the volume is what hurts, because the bot answers hundreds of shipping questions a week.
The honest metric is not deflection. It is the share of chatbot shipping answers that match the real policy, checked by sampling conversations against the live rates. Most stores have never run that check, because the chatbot vendor reports containment, and containment looks like success until the support queue is read.
Fixing it without removing the bot
Ground the bot in live data. Shipping questions should hit the same rates API the checkout uses, with the shopper's postcode, not a language model guessing from a help article. If the bot cannot look the rate up, it should say so and hand the shopper to the shipping page or a human, instead of inventing a date.
Then audit weekly. Sample twenty shipping conversations, check each answer against the live policy, and log the error rate. When the error rate is above zero, the bot is a liability in that category, and the fix is either better grounding or removing shipping from the bot's scope entirely.
Related reading: AI Commerce Watch, Answers, Glossary.
Frequently asked questions
Why do AI chatbots get shipping questions wrong?
Because shipping is a live lookup, not a conversation. The right answer needs the postcode, the cart contents, the warehouse, and the carrier cutoff, and a bot answering from training data or an old help article will give the average answer, which is wrong for most postcodes.
What does a chatbot shipping error cost a store?
Usually a refunded shipping fee or a reshipped order per incident, plus the trust of a shopper who was promised a delivery date the store never offered. The cost is small per case and large in volume, because the bot answers hundreds of shipping questions a week.
How should a store measure chatbot accuracy?
Sample conversations and check each answer against the live policy, instead of trusting the vendor's deflection rate. The useful metric is the share of shipping answers that match the real rates, checked weekly.
Should a store remove the chatbot from shipping questions?
Not necessarily. Ground it in the same live rates API the checkout uses, with the shopper's postcode. If the bot cannot look the rate up, it should hand off to the shipping page or a human instead of inventing a date.