We asked 40 live support bots what their customers ask. Here is what came back

This month we did something simple at a larger scale than usual. We picked 40 live AI support bots on business websites in the UK, the US, Ireland and Australia, and asked each one the questions an ordinary customer would ask: prices, delivery, returns, schedules, opening hours, what’s included. No tricks, no attempts to make the bots misbehave, no personal data, no requests for a person. Then we compared every answer with what the business’s own website said.

The results, in three numbers:

	Bots
Gave at least one wrong fact about the business itself	19
Had at least one problem worth fixing (wrong fact, missed answer, loop, broken flow)	26
Got everything right	14

Nineteen out of forty is a lot. It’s also not a verdict on AI assistants: fourteen bots answered everything correctly, which shows it can be done. What separates the two groups is rarely the platform. It’s how the bot was set up and whether anyone checks it.

We won’t name or describe any of the businesses; we may write to some of them. Below, each pattern is retold with a made-up business from a different trade, keeping only the mechanism.

1. The neighbouring row

The most common wrong answer. The bot finds the right price table and reads the wrong row: the larger size, the other plan, the weekend rate. The answer quotes a real price from the real page, which is why nobody notices.

Retold: a hotel’s bot is asked the price of a standard double for two nights and gives the price of the superior double from the next row. The guest books expecting one price and sees another at checkout. More on this on wrong prices.

2. The invented rule

A fee, a deadline or a policy the business never wrote. Usually a plausible general rule filling a gap where the bot couldn’t find the business’s own.

Retold: a language school’s bot says cancellations are refundable “up to 48 hours before the course starts”. The school’s terms say 7 days. We explain the mechanism in why chatbots invent return policies.

3. “We don’t have that”

The bot says something isn’t offered, or that it can’t share the information, when the answer is printed on the site: on the pricing page, in the FAQ, sometimes on the home page.

Retold: a coworking space’s bot says there are no day passes. The pricing page lists one. Often the answer sits in a table, an image or a tab the bot never read. See denies published facts.

4. The answer to a different question

Asked about one product, the bot answers about another with a similar name. Asked about delivery, it shows a product carousel.

Retold: a car rental bot asked about the deposit for a van answers with the deposit for cars.

5. Loops, silence and broken flows

A typing indicator that never ends, the same reply to every question, a form that comes back after each answer, or an error message after an update nobody noticed. Fewer bots, but every visitor hits them.

6. Details before answers

The bot asks for a name and email before answering a question the FAQ answers in public. We stopped the conversation there in every case; many real visitors do too. See data before answers.

What the 14 clean bots had in common

We can’t see inside anyone’s setup, but the bots that got everything right shared visible traits:

• Narrow, current sources. Their answers matched the current pages, never an old version.
• Honest “I don’t know”. When a question wasn’t covered, they said so and offered a way to reach the team, instead of guessing.
• Conditions kept. Prices came with their unit and plan; rules came with their exceptions.
• No gates. They answered first and asked for details only when a hand-over needed them.

None of that depends on which vendor’s bot it was. It depends on content and on checking.

What to do with this

If you have a bot, ask it ten questions your customers ask, and compare each answer with your own page. If one in ten is wrong, you’re in the 19. The fixes are usually in the content: what to put in the knowledge base is the place to start, and our 25-question checklist is a ready-made set.

The other conclusion is about measurement. Most of these bots would look fine on their own dashboards: conversations ended, customers didn’t ask for a person. That’s the gap between resolution rate and accuracy.

If you’d like your bot checked the same way, every week, that’s Assistant Care. The method follows the testing advice the platforms give themselves; Zendesk, for example, recommends testing with people who didn’t build the bot.¹ Intercom’s guidance on content describes the source problems behind most of what we found.²

Method: 40 bots on business websites in the UK, US, Ireland and Australia, October 2026. Five to six ordinary customer questions per bot, typed as a customer would, compared with the business’s own published pages. No personal data given; no hand-over to a person requested.

Footnotes

1. Zendesk, “Best practices for testing AI agents”, support.zendesk.com. ↩

2. Intercom Help, “Optimizing content for Fin”, intercom.com/help. ↩