AI hallucination rates: how often AI makes things up

Ask how often AI hallucinates and you'll be quoted a single figure. Ignore it. Published rates run from 1.8% to 88%, and the spread comes from the task, not from sloppy research.

Read down the table and a pattern appears. When the model has a document in front of it, errors are in single digits or low teens. When it works from memory on specific, checkable facts, error rates can pass 50%. When the facts are obscure (a case holding, a citation, a named person), they climb further.

So the honest answer to "how often does AI hallucinate?" is a question back: doing what, with what in front of it?

OpenAI doesn't publish an overall error rate for ChatGPT, and nobody else can, because it would depend on what everyone asks it. What exists are benchmark rates and audits.

OpenAI's own numbers. In its GPT-5 system card, OpenAI reported that on production-style traffic with browsing on, gpt-5-main's claim-level hallucination rate was 26% lower than GPT-4o's, and gpt-5-thinking's was 65% lower than o3's. On SimpleQA without web access, gpt-5-thinking still hallucinated on 40% of questions (OpenAI GPT-5 system card, August 2025). These are relative improvements and benchmark-specific figures, not a percentage of everyday answers that are wrong.

Newer isn't always better. OpenAI's o3 hallucinated on 33% of PersonQA questions, about double o1's 16%. OpenAI's explanation was that o3 "tends to make more claims overall", so it makes more correct claims and more wrong ones, and it said "more research is needed" (o3 and o4-mini system card, April 2025).

Independent audits of ChatGPT with search. Asked to identify the source of 200 news excerpts, ChatGPT Search got 134 wrong and declined none (Tow Center, March 2025). An EBU and BBC study of 3,000+ answers from ChatGPT, Copilot, Gemini and Perplexity found 45% had at least one significant issue, and 20% had major accuracy problems (EBU, October 2025).

Our reading: for a fact recalled from memory, assume something like a coin flip on obscure details and much better odds on common knowledge. For anything with a source attached, assume the source might not say what the answer says.

The model you pick matters. These matter more.

1. Whether it has the source. The gap between Vectara's single-digit rates and PersonQA's 33 to 48% is mostly the gap between "here is the text" and "tell me from memory". FinanceBench shows it inside one model: GPT-4-Turbo answered 79% correctly with the whole filing in its context, but only 19% using a shared retrieval system (Patronus AI, 2023).

2. Whether guessing is rewarded. OpenAI's researchers argue that hallucinations persist because training and evaluation "reward guessing over acknowledging uncertainty" (Kalai et al., September 2025). A model tuned to always answer will always answer.

3. How obscure the fact is. Stanford found that on a case's core holding, general models hallucinated "at least 75% of the time" (Stanford HAI, January 2024). Specific names, dates, citations and figures are where memory fails first.

4. How long the task runs. In a study of 14 models, factual accuracy fell by about 42% as tool calls grew from 2 to 150 (Onweller et al., Cited but Not Verified, May 2026). Long agent runs and deep-research reports give errors more places to creep in.

If you control those four, you control most of your risk. Our guide to prompts that make AI cite sources covers the first two.

A 5% error rate sounds tolerable until you multiply it by every brief, report and filing that leaves the building.

The legal profession keeps count. Damien Charlotin's database lists 2,202 court and tribunal decisions where a party relied on hallucinated content (last updated 19 October 2026). The early case, Mata v. Avianca, ended in a $5,000 sanction for six invented ChatGPT cases in 2023. In Couvrette v. Wisnovsky, an Oregon magistrate imposed about $110,000 in fines and fees and dismissed the case (ABA Journal, April 2026).

Outside court: Deloitte agreed to partially refund an A$440,000 contract after its report for the Australian government contained a fabricated quote from a court judgment (AP, October 2025). Air Canada was held liable for its chatbot's wrong advice on bereavement fares, with a total award of CAD $812.02; the tribunal rejected the argument that the chatbot was a separate legal entity (Moffatt v. Air Canada, 2024).

None of these happened because a model had an unusually bad rate. They happened because nobody checked the output before it went out. We've written about why polished output gets checked least in the hidden AI hallucination.

Published rates tell you about someone else's questions. Yours are probably different, and you can measure them on the next answer you were about to use.

A two-minute check

1. Take one real AI answer you were planning to send, paste or act on.
2. Underline every checkable claim: each number, name, date, quote and "according to". A few paragraphs can easily hold ten.
3. Check three at random against the source: open the link, the document or the data.
4. Sort each into right, wrong, or unsupported (it might be true, but nothing shown backs it).
5. Count the unsupported ones as risk, not as fine. That's where most hallucinations hide.

Three claims won't give you a statistically meaningful rate. They will tell you whether this kind of answer, on this kind of question, needs checking every time. Usually it does.

Checking every claim by hand is the slow part. Count connects to Claude or ChatGPT through its MCP server, splits the answer into claims and labels each one: Citation, Assumption, Model knowledge or Hallucination, with the reasoning chain that shows where an unsupported claim came from. For the full manual method, see how to check AI answers.

There's real progress. GPT-5-thinking's 65% reduction against o3 on production-style traffic is a large drop, and the best summarising models on Vectara are now under 4%.

But progress is uneven across tasks, and leaderboards lag releases: the newest Claude 5.x, GPT-6 and Gemini 4 models weren't on Vectara's board when we checked. More capable models also attempt more, which can push their error counts up even as their accuracy rises, as o3 showed.

We don't expect hallucinations to reach zero. Kalai and colleagues argue some are a statistical consequence of how models are trained, which is a reason to change how they're scored, not a reason to expect them to vanish. Plan for a rate above zero, and make the wrong claims easy to find.

How often does AI hallucinate?

What is ChatGPT's hallucination rate?

Which AI hallucinates the least?

Why does AI hallucinate?

Will AI hallucinations ever be fixed?