A common chatbot mistake
Your chatbot makes up fees, policies and deadlines you never wrote
A delivery charge, a refund window, a fee: confident, specific and nowhere on your site. This is the answer that ends up in a dispute.
When the bot can’t find an answer, it rarely says so. It writes one that sounds right: a fee in the usual range, a refund window like other shops have, a deadline that seems reasonable.
These answers are the hardest to spot from the inside, because they read exactly like your real policies.
Is there a charge for a second driver?
Adding a second driver costs $15 per day.
Your page says
Second driver: free on every rental.
Rental terms, extras section
What if I cancel my desk booking on the day?
Same-day cancellations are charged 50% of the booking.
Your page says
Cancel before 9:00 on the day for a full refund. There is no cancellation fee.
Booking terms
Why it happens
It isn’t random
The model fills the gap
Language models are built to answer. If the knowledge base has nothing on the question, many bots still produce a plausible reply instead of saying they don’t know.
Two half-answers become one wrong one
The bot joins pieces from different pages: a fee from one service, a condition from another.
The instructions don’t forbid it
Most bots are set up with a friendly tone and no rule about what to do when the answer isn’t in the sources.
What it costs you
- An invented refund window or fee is a promise in writing. Customers screenshot it and quote it back.
- Invented charges put people off before they buy, and you never learn why.
- Air Canada was ordered to pay a passenger the bereavement discount its chatbot described wrongly. The tribunal rejected the idea that the bot was responsible for its own words.
How we check
- 01We ask about the things customers ask but businesses often don’t publish: delivery to remote areas, refunds on sale items, cancellation the day before.
- 02Every policy-like statement the bot makes is traced back to a page. If there is no page, it is an invented policy.
What the fix looks like
- Publish the missing answers, so the bot has something true to say.
- Instruct the bot to say it doesn’t know and point to the contact page when the sources are silent.
- Re-test with the same questions plus variations.
Where we see it
Other mistakes we find
Wrong prices
The bot finds the right table and reads the wrong row: the pricier room, the member rate, the other plan. Customers act on it.
Read more“We don’t have that”
The answer is on your pricing page, your FAQ, even your home page. The bot says it can’t share it, it isn’t announced, or it doesn’t exist.
Read moreAnswers a different question
Asked about one product, it answers about another. Asked about delivery, it shows a product carousel. The customer has to start again, or gives up.
Read moreLoops and silence
A typing indicator that never ends. The same reply to every question. A consent screen that comes back after every click. The customer can’t get past it.
Read moreBroken after an update
An expired key, a changed model, a moved page. The bot shows an error to every visitor, and it can go on for days.
Read moreName and email first
Asked a question your FAQ answers in public, the bot asks for a name, email or phone number first. Most visitors close the chat.
Read moreSee your assistant answer your customers first
Email us your store’s address. Within a working day you get five questions your customers ask and the answers your assistant would give, each with the page it comes from. Free, no call needed.