Finance agents can match analysts on the number and still miss the judgment

Matching the consensus, missing the range

In a preprint, EPSBench evaluates LLM agents on forecasting quarterly earnings by linking financial evidence to income-statement drivers. Agents read a fixed collection of filings, earnings-call transcripts, and news, and a shared accounting calculator converts their estimates into GAAP diluted earnings per share. The benchmark covers 149 firm-quarters from 86 US companies and compares six models against analyst consensus. Three models showed no detectable paired difference in point forecast error from consensus, and three had higher error. Errors in operating-income estimates explain most of the performance differences between models. m:…6693

The headline result is the calibration failure. Nominal 80% prediction intervals contained reported EPS in only 37% to 70% of runs across models, and most prediction interval misses fell above the upper bound because the forecasts ran low. An agent that matches consensus on the point estimate but whose 80% interval misses more than half the time is not a forecaster a finance team can rely on for risk decisions; it is a point-estimate machine with false confidence. The benchmark's contribution is making that gap measurable rather than anecdotal. m:…6693

Good at research, weak at judgment

In a second preprint, the Investor's Last Exam benchmark evaluates financial agents on end-to-end equity research using frozen, timestamped evidence, expert criteria, and dependency graphs, shifting the evaluation unit from isolated capabilities to the evidence-assumption-judgment chain. It uses a versioned financial database containing 3.43 million document versions across nine source categories and 86 professional assignments, with historical timestamps designed to prevent models from seeing the future. GPT-6 Astra scored 68.1 out of 100 overall and Claude Opus 5.5 scored 66.9, while DeepSeek V4.1 Flash outperformed Astra on realized outcomes, scoring 40.3 versus 37.7. m:…6595

The split inside Astra's score is the finding: 78.9 on Research & Evidence against 52.5 on Forecast & Model. The benchmark's authors describe current models as very good at organizing evidence and not yet good at turning that analysis into judgment. That matches EPSBench from a different angle: the research and the point estimate are now within reach, but the judgment step, the forecast, the range, the willingness to be uncertain, is where the chain breaks. The benchmark is also designed as a training tool, with its timestamps usable as a re-samplable environment for large-scale domain reinforcement learning, so the gap it measures is one its authors expect to be trained against. m:…6595

What finance teams actually do

DualEntry's post on AI in FP&A describes a profession that has, so far, acted as if it already understood these limits. As of July 2025, 81% of FP&A professionals were using AI in their work, but only 26% were using it regularly, and 53% of senior finance leaders reported not using AI in their FP&A function in early 2025. AI is most useful, the post argues, when tasks are repeatable and underlying data is available, and it is currently focused on report writing and Excel automation rather than core forecasting. Human finance professionals must still confirm the causation behind financial variances identified by AI, and AI cannot replace the business partnering role, which requires judgment, stakeholder context, and challenging assumptions. m:…8402

The post also notes that 63% of organizations lacked or were unsure about having the right data management practices for AI as of July 2024, that AI performance in accounting tasks varies significantly with AR model scores ranging from 41.7% to 91.7% as of October 2026, and that AI agents in finance should be piloted on low-risk, contained workflows so errors are visible and reversible. DualEntry argues its agentic ERP helps by letting AI access financial context directly from the ledger rather than static data copies. The long-term impact of agentic ERPs on finance headcount remains to be seen, the post concedes. The picture is consistent: finance is using AI where errors are cheap and visible, and keeping judgment where errors are expensive. The benchmarks suggest that is the right order, for now. m:…8402