Experts are deferring to agents without noticing. Accountability has to be built, not assumed

The deferral finding

Anthropic study of expert judgement is uncomfortable reading for anyone who assumes a human in the loop is a safeguard by itself. Professional analysts deferred to their AI agents preferences 86% of the time across four analytical decisions, according to the researchers. In post-task explanations, 77% of analysts repeated their agent arguments, while only 15% credited the agent as the source. When analysts had an existing preference, they switched away from it 60% of the time if the AI agent disagreed, compared with 6% if it agreed. Analysts who followed the agents recommendations closely were as confident in their final recommendation as those who did not. The study used Claude Opus 4.8 running at High effort for the AI agents, and the researchers deliberately strengthened the agents preferences to measure their effect on analyst decisions, so the headline rate is a product of that design and should not be read as a natural rate in every workplace. m:…1916

The study is not a simple warning against using agents. Analysts who worked without an AI agent were slower and made significantly more methodological errors than those who used one, and 55% of participants who worked without an AI agent made serious methodological errors, compared with 6% of those who used one. The researchers suggest that experts often form their views while working, making them susceptible to adopting an agent prior beliefs, and that if separate research teams all defer to models with the same preferences, their agreement may no longer be independent. That second point is the policy problem in miniature: independent-looking agreement can be a shared model preference wearing different letterheads. The researchers suggest that AI agents could be designed to ask experts for their own view before offering one, though this could be overwhelming for users. They also suggest that institutions need an explicit strategy for establishing the role of humans and the development of their skills as AI capabilities increase. m:…1916

Shared accountability, in plain terms

Goldman Sachs chief executive David Solomon framed the accountability question on a podcast in terms that avoid the usual binary. He said the current AI debate is too binary, focusing on a choice between innovation without guardrails and heavy-handed regulation, and that responsibility for AI control should be distributed among governments, model developers and the businesses deploying the technology. Goldman Sachs has utilised AI-type models in trading platforms for nearly 20 years with human accountability attached to every platform, according to Solomon, and financial institutions can provide valuable risk management and audit practices to help other businesses safely deploy AI. The claim is not that banks have solved accountability, but that they have a long practice of attaching a named human to a system that makes consequential decisions. m:…3249

Checkpoints in practice

Finance offers a concrete version of that distribution. Financial institutions are using agentic AI to parse technical files, draft questionnaire answers and propose initial risk tiers, according to Domino Data Lab, and modern enterprise architectures pair agentic document parsing with deterministic rule engines. Governance of agentic intake requires containment, tracing and reproducibility, and human review controls. The workflow consists of four steps: Read, Match, Flag and Log. Financial institutions rarely rely on AI agents to perform mathematical tier calculations directly due to output variance. That division of labour is the policy in miniature: let the agent read and propose, let deterministic rules and a human decide the number, and log every step so the decision can be reconstructed later. m:…8969

Regulatory context in the same write-up shows why that division matters. SR 26-2 applies to US banks with over $30 billion in assets, according to Domino Data Lab, and the EU AI Act classifies AI systems based on intended use, with creditworthiness and insurance pricing systems designated as high-risk. In the public sector, AWS Jake Burns makes a parallel point about adoption rather than regulation: agentic AI is defined by generative models having access to tools that allow them to take action in the world, organisations fail to move AI from pilot to production because they lack a bridge phase that replicates production-like messiness, and workforce buy-in is the most critical factor for successful AI adoption in large organisations. The most successful AI implementations in large enterprises are simple, multi-step workflow automations rather than fully autonomous systems, according to Burns. Both examples point the same way: start with bounded workflows, keep a human checkpoint, and treat buy-in as a design requirement rather than a communications task. m:…8969 m:…5771

What to design for

This suggests the oversight question is not whether a human is somewhere in the process, but whether that human can actually disagree. If 77% of analysts repeat agent arguments while only 15% credit the agent, then a review step that asks for a final sign-off will mostly rubber-stamp the agent view. A stronger design asks for the human view first, logs the agent proposal separately, and keeps deterministic checks for any calculation that affects a person money or a regulated decision. Governance that requires containment, tracing and reproducibility, and human review controls, as Domino Data Lab describes, is a checklist a team can actually audit, rather than a promise that a human was involved. m:…1916 m:…8969