How to check if an AI answer is right, one claim at a time

"Is this answer right?" is the wrong question, and it's why checking AI feels impossible. A five-paragraph answer isn't one thing. It's a dozen or more separate statements, some sourced, some calculated, some quietly assumed, glued together with fluent prose.

Researchers who measure factual accuracy worked this out early. FActScore, from 2023, splits long-form text into atomic facts and checks each one against a knowledge source; by that measure, ChatGPT's biographies were only 58% accurate (Min et al., EMNLP 2023). Google DeepMind's SAFE does the same with search and agreed with human annotators 72% of the time (Wei et al., 2024).

The unit is the claim. An answer with eleven right claims and one wrong one isn't 92% right; if the wrong one is the number your recommendation rests on, the answer is wrong where it matters.

Courts have arrived at the same place. In June 2025 the High Court of England and Wales said lawyers have a professional duty "to check the accuracy of such research by reference to authoritative sources" before relying on AI output (Ayinde v Haringey; Al-Haroun v QNB). Checking against a source means checking claim by claim. There's no other way to do it.

Three steps. The first two take a minute; the third is where the time goes, so the first two exist to make it shorter.

1. Split

Go through the answer and mark every statement that could be true or false: each number, name, date, quotation, comparison and "because". Leave out the connective tissue ("this suggests", "in summary").

2. Sort

Put each claim in one of five piles, by what it rests on:

3. Check, in order of consequence

Start with the claims your conclusion depends on: the ones that, if wrong, would change what you'd do. Check background claims only if you have time.

The last pile deserves suspicion. Model memory has a cut-off date and no citation. It may well be right, but it's the pile where hallucinations live, and it's the hardest to spot because it reads exactly like the others.

The answer below is illustrative: we wrote it to show the kind of response an assistant might give when asked why churn went up. It isn't output from a real test, and the figures are invented.

Monthly churn rose from 2.1% in June to 3.4% in August. The increase came mainly from customers on the legacy Starter plan, following the July price rise. Healthy SaaS businesses keep monthly churn below 2%, so this needs attention. Reverting the Starter price would recover about $18,000 in monthly recurring revenue.

Four sentences. Here are the claims inside them.

Five claims, four different checks. Claim 3 is the one the recommendation hangs on, and it's an assumption presented as a finding. It's the pattern to look for first: the numbers check out and the "because" was never tested.

Some habits give the feeling of verification without the substance.

• Asking the model "are you sure?" It can change a right answer to a wrong one to please you, or defend a wrong one fluently. Either way you've learned about its tone, not the fact.
• Asking a second model. Two models trained on similar text can share the same wrong belief. Agreement is weak evidence; a source is strong evidence.
• Seeing that the link opens. In a 2026 study, the best models' links worked more than 94% of the time while only 39 to 77% of claims were accurate (Onweller et al., Cited but Not Verified). An earlier study found only 51.5% of search-engine sentences were fully supported by their citations (Liu, Zhang and Liang, 2023).
• Reading it carefully. Fluency hides errors. We've written about that effect in the confidence bubble.

One model-side technique does help, as a signal: run the same prompt more than once and compare. Anthropic notes that "inconsistencies across outputs could indicate hallucinations" (Claude docs, Reduce hallucinations). Consistency doesn't prove a claim, but inconsistency is a good reason to check it first.

Most answers don't justify a full audit. These five steps fit in two minutes and catch the errors that do damage.

A two-minute check

1. Read the conclusion and underline its one or two "because" claims. These are usually assumptions.
2. For each, ask what you'd expect to see if it were false, and spend thirty seconds looking.
3. Pick the most important number and trace it: open the source or re-run the query.
4. Open one citation at random and find the exact sentence it supports.
5. Ask the assistant: "List every claim in your answer and say whether it comes from a source, a calculation, my data, or your own knowledge."

The full version is the split-sort-check method above, on every claim. It's slow. A Datarails survey found finance teams spend 26% of their working week checking AI output (vendor survey, October 2026). Much of the work is the splitting and sorting before any checking starts.

That's the part Count does. Connect it to Claude, ChatGPT or any assistant that supports MCP, ask it to audit an answer, and it replies with a link to a canvas holding every claim. Each one is labelled Citation, Assumption, Model knowledge or Hallucination, with a reasoning chain showing the prompt, what the model already believed, each search, query or calculation, and the source the claim rests on. When a step is wrong or missing, the chain shows where. In the churn example, claim 3 would surface as an assumption, which is the one you'd want to see first.

Not every answer needs this. A draft email, a brainstorm or a summary you'll read the source for anyway can go unchecked.

The line we use: check claim by claim when the answer will be repeated (sent, published, presented) or acted on (a decision, a filing, a number in a model). Those are the cases where someone else will trust it because you did. The 2,202 court decisions in Damien Charlotin's database of AI hallucination cases (October 2026) are, almost by definition, stories of an answer that crossed that line unchecked.

We've written more on why trust in AI depends on seeing its working in the AI trust gap. For how often answers need fixing in the first place, see AI hallucination rates.

How do I check if ChatGPT is right?

Can AI fact-check itself?

Is there an AI fact-checking tool?

How do I verify AI sources?