Why bots sometimes get things wrong, and how to check their work Explainer

3 min read

AI bots are fast and helpful, but they can still misread a request or state something that isn't true. Here's why that happens and a few quick habits for catching it.

The short version

An AI bot writes its answers by predicting what a good answer should look like. Most of the time that works well. Sometimes it fills a gap with something that sounds right but isn't, like a date, a number or a name. People call this a hallucination. It isn't lying; the bot simply doesn't always know when it's guessing. The fix is easy: check the parts that matter before you rely on them.

Why mistakes happen

• Missing information. If the bot can't see the email, file or web page that holds the answer, it may guess instead. Connecting the right app, or pasting the text in, helps a lot. See What is a connector.
• A vague request. "Total up the orders" could mean this week, this month or all time. The bot has to pick one. See How to write a request your bot gets right the first time.
• Out-of-date knowledge. A language model learned from text up to a certain point. For anything recent, like prices, news or a company's current hours, it needs to look it up rather than remember it.
• Long or messy input. In a huge document or a long thread, small details are easier to miss.

What to check, and what to skip

You don't need to check everything. Match the effort to the stakes:

• Check closely: money, dates, addresses, names, quotes, anything sent to a customer, and anything you can't undo.
• Spot-check: summaries and lists you'll act on. Open two or three of the original items and compare.
• Skim: brainstorms, first drafts and ideas. You're going to edit those anyway.

Five quick habits

1. Ask where it came from. "Which email did that come from?" or "Give me the link you used." A real answer points to something you can open.
2. Compare against the original. For a summary of an inbox or a sheet, open a few of the real items and make sure they match.
3. Re-check the numbers. Ask the bot to show how it got a total, or add up a few rows yourself.
4. Let it say "I don't know". Add "If you're not sure, say so instead of guessing" to important requests.
5. Draft before sending. Anything that goes to another person should be a draft you read first. More in approval before sending.

When you find a mistake

Say what's wrong in the same chat, plainly: "The total is off; order 1042 was cancelled." The bot can correct it with that context. If the same mistake keeps coming back, ask it to remember the fix (see Memory), or save the corrected steps as a skill.

Where to go next

Read Staying safe with AI bots for the habits that keep a bot in check when it can send, buy or delete things.