The best AI for financial analysis still gets one task in three wrong

Vals AI built its Finance Agent benchmark from 537 questions of the kind a junior analyst handles: retrieving a figure from a filing, researching a market, building a projection. The top score is 64.37%, from Claude Opus 4.7, followed by Claude Sonnet 4.6 at 63.33%, Muse Spark at 60.59% and DeepSeek V4 at 60.39% (Vals AI, v1.1, updated 4 June 2026).

Put plainly, the best AI analyst on that test got roughly one task in three wrong. That isn't a reason to avoid AI in finance. It's a reason to know which third.

Two things to bear in mind when reading that board. It was last updated in June, so models released since, such as Claude Opus 5.5 (September 2026), can't be on it. And secondary sources report a newer version 2 of the benchmark whose scores aren't comparable with v1.1; when we checked on 22 October 2026, Vals AI's page still showed v1.1. Other Vals finance boards tell a more cheerful story, but they're narrower: on CorpFin, which tests long credit agreements, Claude Opus 5 leads at 73.19% (August 2026), and Vals has archived both CorpFin and TaxEval v2 as saturated.

The most quoted finance figure is also the most misquoted. FinanceBench, from Patronus AI, reported that GPT-4-Turbo with a retrieval system "incorrectly answered or refused to answer 81% of questions" about public-company filings (Islam et al., November 2023). It's often repeated as "AI is wrong 81% of the time". It isn't what the study found.

Most of the 81% was refusal: the retrieval system couldn't find the right passage, and the model said so. The more worrying row is the last one. Given the exact page with the answer on it, the model still got 15% wrong.

That pattern, retrieval failures dominating and reasoning errors persisting even with perfect inputs, is the most useful thing FinanceBench tells you. The study is three years old. The pattern still shows up in newer work: in Vals AI's 2025 legal report, AI tools beat lawyers on document Q&A and summarising but lost on the tasks that needed source retrieval and precise edits (Vals Legal AI Report, February 2025).

So when you see a finance accuracy figure, ask three things: was the model given the documents, did refusals count as failures, and how old is it?

Finance work punishes small errors, because one wrong cell feeds every cell downstream. These are the failure points we'd watch:

• Pulling the wrong number. The right metric from the wrong period, segment or restated version. FinanceBench's retrieval row is this failure at scale.
• Doing maths from memory. Multi-step arithmetic without code is unreliable; use the assistant's code execution, and check the formula, not just the output.
• Picking a definition silently. Adjusted or reported, fiscal or calendar year, gross or net. The model will choose one. It may not tell you which.
• Being swayed by irrelevant detail. Adding one irrelevant clause to maths problems cut GPT-4o's accuracy from 95.2% to 63.1% (Apple, GSM-Symbolic, 2024). Filings and board packs are dense with detail that doesn't matter to your question.
• Inventing a source. Deloitte agreed to partially refund an A$440,000 government contract after its report was found to contain a fabricated quote from a court judgment; the revised report disclosed that GPT-4o had been used (AP, October 2025).

Finance teams know this already. In a Datarails survey of 270 US finance leaders, 65% said their most common frustration with AI was "confident answers built on the wrong data", and only 5% trusted it to produce board-ready reports without review (Datarails, October 2026; vendor survey). The same respondents reported spending 26% of their working week checking AI output.

Choose a finance AI the way you'd test a new analyst: give it work you've already done and compare.

1. Take a recent 10-K or annual report you know well, and five questions you've answered from it: a segment revenue figure, a year-on-year change, a covenant term, a calculated ratio, and one question the filing can't answer.
2. Run each candidate with the filing attached, not from memory. FinanceBench showed the gap between having the document and not is enormous.
3. Score right, wrong and declined separately. The unanswerable question is the one that matters; a tool that invents an answer to it will invent others.
4. Check the cited page for each right answer. A right number with a wrong citation is luck.

On connectivity, Anthropic's Claude for Financial Services links Claude to market data and internal data through partners including FactSet, Morningstar, S&P Global, PitchBook, Snowflake and Databricks, and says claims link back to their original sources (Anthropic, July 2025). Even Anthropic's own tutorial notes that Excel models "may require manual review and adjustment for complex formulas" (Claude Academy).

Our position: for finance, connected beats clever. A slightly weaker model reading your actual ledger is more accurate than a stronger one working from a pasted extract. For prompts that state definitions and periods up front, see prompts for financial analysis.

Datarails' respondents reported spending a quarter of their week checking AI output. Most of that time goes on finding which number to check. This is the short version.

A two-minute check

1. Find the three figures the conclusion depends on. Revenue growth, margin, the projection's key assumption. Ignore the rest for now.
2. Trace each to its page. Open the filing at the cited page and find the number. Confirm the period and whether it's adjusted or reported.
3. Recompute one derived figure (a ratio, a growth rate) from the underlying numbers yourself.
4. Ask the assistant: "Which numbers did you calculate, which did you read from a document, and which are assumptions?"
5. Flip one assumption. If the growth rate was assumed, change it and see whether the conclusion survives.

Count makes step 4 automatic. Connect it to Claude or ChatGPT and it labels every claim as a citation, an assumption, model knowledge or a hallucination, and shows the reasoning chain: the document, query or calculation each number rests on. When an assumption is buried three steps down, the chain shows where. For the general method, see how to check AI answers.

What is the best AI for financial analysis?

Can ChatGPT do financial analysis?

How accurate is AI for finance?

What are the best AI tools for finance teams?