When agents fail, the best auditor is the agent itself
POIROT beats centralised LLM evaluators by up to 50 points at blaming the right agent. The catch: when agents talk to each other about the failure, they converge on the wrong culprit.
- In a preprint, the POIROT protocol outperforms centralised LLM evaluators in failure attribution across all nine tested model-benchmark configurations, by up to 50 percentage points. Self-reported m:…9318
- In a preprint, independent self-assessment is the primary driver of POIROT's performance, while peer interaction phases can degrade accuracy by causing convergence on incorrect culprits. Self-reported m:…9318
- In a preprint, the GraphHarness mechanism improves task success by 48.5, collaboration quality by 30.6, and reduces costs by 34.0 compared with standard baselines in decentralised multi-agent collaboration. Self-reported m:…9321
- In a preprint, evidence-guided recovery certified 30 completions versus 9 for evidence prompting across 58 paired units, a completion rate difference of 36.2 percentage points. Self-reported m:…2589
- In a preprint, the Evidence-Governed Evolution framework separates candidate generation from confirmatory adoption to prevent degradation, and uses reference-relative non-inferiority tests to protect declared capabilities. Self-reported m:…9305
Who broke it?
When a multi-agent system fails, the first question is which agent broke it, and the usual answer is to ask a bigger model to judge. A preprint this week says that is the wrong approach. POIROT — phased oversight via introspection, review and opinion tallying — gives agents a decentralised protocol for failure attribution, and the authors report it outperforms centralised LLM evaluators across all nine tested model-benchmark configurations, by up to 50 percentage points. They also release BLAME, a benchmark for fault localisation in high-stakes domains such as clinical rehabilitation and algorithmic trading. The auditor is not a supervisor above the system. It is the system, auditing itself. m:…9318
The finding that makes the paper interesting is the one that cuts against instinct. Independent self-assessment is the primary driver of performance in POIROT, the authors report, while the peer interaction phases — agents sharing evidence and interrogating each other — can degrade accuracy by causing convergence on incorrect culprits. Agents that talk to each other about a failure agree with each other, and the agreement is often wrong. It is a result that should be read alongside the week's other multi-agent research, and it sets a design rule: let each agent assess itself first, and treat consensus as a suspect, not a signal. m:…9318
Decentralised, but coordinated
A second preprint, DecArena, looks at the coordination problem underneath. Decentralised multi-agent systems face scalability and privacy concerns when they rely on a global orchestrator, and current LLM agents struggle to discover non-local relevant resources — the thing they need is held by an agent they do not know. The authors propose GraphHarness, a topology-aware routing mechanism, and report it improves task success by 48.5, collaboration quality by 30.6, and reduces costs by 34.0 compared with standard baselines. Standard reinforcement learning, they add, fails to reliably overcome the limitations of decentralised collaboration. The fix is structure, not more training: give the agents a map of who holds what. m:…9321
The third preprint asks what happens when the agents start changing themselves. Evidence-Governed Evolution governs how autonomous agents adopt self-generated modifications to their prompts, tools and workflows, by separating candidate generation from confirmatory adoption to prevent optimism bias — the agent that proposes a change is not the one that approves it. Reference-relative non-inferiority tests protect declared capabilities during evolution, and the safety guarantees are conditional on bounded paired scores, fresh sampling and valid reference-evaluation constructs. Self-improvement, in other words, is allowed, but only through a gate that can say no. m:…9305
Recovery and privacy
The last two papers handle the mundane failures that actually fill an operations log. Tool timeouts can leave AI agents facing ambiguous outcomes and partial execution — did the payment go through, or did the call just die? An evidence-guided recovery framework introduces a terminal certificate that separates actual goal satisfaction from publicly supported safe completion, and across 58 paired units in 19 task families it certified 30 completions versus 9 for evidence prompting, a completion rate difference of 36.2 percentage points. The guarantees are tied to declared provider capabilities rather than assuming third-party exactly-once execution, which is the honest version: the system promises only what the provider actually declares. m:…2589
GuardianAgentFlow closes the week with the privacy half of the same problem. It mediates outgoing requests in web-based AI agents through explicit release contracts, with a restriction check and a candidate-grounded feedback loop to protect private context without blocking the task. The authors report it improves request-contract success rates compared with baseline methods, with lower mediation latency than full-policy reasoning, and provides complementary protection when composed with existing privacy-aware agents such as PrivacyLens and AgentDAM. The week's arc is consistent: the multi-agent system is getting its own internal institutions — an auditor, a map, a gate for self-change, a certificate for completion, and a contract for privacy. None of them is a bigger model watching from above. All of them are built into the system itself. m:…9304
What is still unresolved
- Whether independent self-assessment holds when agents share a flawed world model
- Whether topology-aware routing scales to hundreds of agents
- Whether self-modification gates hold under adversarial prompts from inside the system
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.