The judges disagree: evaluating long-horizon agents is now the hard problem, not building them

The evaluation problem

In a preprint this week, researchers compared three ways of evaluating AI agents and found they do not agree with each other. The paper finds LLM judges unreliable for detecting forbidden database writes in agentic workflows. Rule-based boundary evaluators outperform LLM judges in identifying forbidden writes. Common agentic gates, verification, correction and safe-termination, do not significantly reduce forbidden write rates,. The uncomfortable conclusion is that the oversight layer many teams rely on is the least reliable part of the system, at least for safety-critical operations. m:…9313

A second preprint explains part of why judging is hard. Search agents often answer confidently despite unreliable retrieved evidence, and may rely on memorized knowledge when documents are factually wrong. The study introduces DeepStress, a dynamic stress-testing framework that subjects agents to degraded evidence at controlled rates, replacing the retrieval module with a synthetic environment where documents are generated dynamically for each query, with independent control of document trustworthiness, relevance and factuality. Robustness of search agents to poor-quality evidence remains under-explored, poor-quality evidence is rare in benchmark corpora, and failures in search agents are hard to reproduce. The authors evaluated 12 systems on questions from HotpotQA and BrowseComp-Plus, and propose two new reliability-aware metrics, RAS and CoPRAS, which credit cautious behaviour and account for computational cost. m:…2833

What works instead

One answer is to judge the steps, not the ending. A preprint on decision models in long-horizon agents finds that a single decision model can function as an actor, a judge or a teacher, and that prior research has primarily evaluated decision models based on their performance as actors. As actors, all six models tested were pulled away from prerequisite subgoals by easier subgoals, but as judges, the same models correctly credited steps that advanced prerequisite subgoals when viewing the full trajectory. Judging whether each step helped is more effective for separating useful from useless actions than predicting task success, and a decision model that acts poorly can still teach reinforcement learning policies effectively. The proposed JEV method improved a Qwen2.5-1.5B policy's success rate on ALFWorld from 72.8 to 82.9 percent and on WebShop from 56.8 to 71.4 percent, using GRPO for training. Judging after the episode ends reveals consequences that acting must anticipate. m:…2584

Another answer is to stop treating agent behaviour as text and start treating it as structure. A preprint on automata from agent traces collapses execution traces into compact finite-state machines, providing a structural substrate for agent behaviour. The resulting machines are compact finite-state machines. FSM-state context outperforms Agent Workflow Memory for next-step prediction, and online monitors using per-state behavioural features can trigger early stopping for failing runs. Behavioural topology, the authors note, is shaped more by the deployment harness than by the language model, which means the harness you ship matters more than the model you pick, at least for how the agent behaves. m:…9314

The multi-agent version of the same problem appears in DecArena, a preprint evaluating large-scale decentralized multi-agent collaboration. Current LLM agents struggle to discover non-local relevant resources in decentralized settings, and decentralized multi-agent systems face scalability and privacy concerns when relying on a global orchestrator. The authors' GraphHarness, a topology-aware routing mechanism, improves task success by 48.5, collaboration quality by 30.6 and reduces costs by 34.0 compared with standard baselines, while standard reinforcement learning fails to reliably overcome the limitations of decentralized collaboration. The numbers are the authors' own, and the units are not spelled out in the claims, so read them as direction rather than precise gains. The direction is that structure, routing by topology rather than by a central brain, beats more training. m:…9321

What to do about it

The practical read for teams shipping long-horizon agents is to stop trusting a single judge. Use rule-based boundary checks for anything that must never happen, like a forbidden database write. Judge trajectories step by step rather than scoring the final answer. Stress-test with degraded evidence, because production evidence is degraded more often than benchmark evidence. And monitor the agent's state machine, not just its logs, so a failing run can be stopped early. All five papers are preprints, and their figures are the authors' own, so treat them as the current best evidence rather than settled science. But they agree on the diagnosis: building the agent was the easy part. Knowing whether it did the right thing is the hard problem now. m:…9313 m:…2584 m:…9314 m:…2833