Agents are learning the hardest skill: when not to act
The best model on a new abstention benchmark scores only 59.5%. Other preprints show agents can skip most helper messages, and that personal-agent updates can be repaired after the fact.
- In a preprint, the best-performing model on the AgentAbstain benchmark (Gemini 3.1 Pro) achieves only 59.5% paired accuracy. Self-reported m:…9758
- In a preprint, certified policies can skip helper messages in 59-80% of cases while retaining 76-95% of performance gains. Self-reported m:…9773
- In a preprint, Kaer shields the planner from untrusted agent feedback by conveying diagnostics to the replanning procedure strictly through abstract templates. Self-reported m:…9778
- In a preprint, automated repair procedures can significantly reduce security violations and increase safe task completion in personal AI agents. Self-reported m:…7398
What changed
Most agent benchmarks measure what an agent can do. A new preprint measures what it refuses to do, and the results are poor. Abstention capability in LLM agents is largely independent of general task-solving capability, so a model that is good at finishing tasks is not necessarily good at stopping. Frontier LLM agents exhibit post-hoc abstention failure modes, where they execute irreversible actions before recognizing abstention triggers. The best-performing model on the AgentAbstain benchmark, Gemini 3.1 Pro, achieves only 59.5% paired accuracy. That is the headline number, and it should worry anyone who lets an agent touch a system where an action cannot be undone. m:…9758
A second preprint looks at restraint from the other side: not refusing a task, but refusing extra work inside one. In multi-agent systems, helper messages are supposed to improve the answer, but many of them add little. Oracle headroom is a weak indicator of message utility, as control inputs reproduce 76% of the oracle's gain, and message skipping effectiveness is determined by the predictability of answer alteration, not the helper agent's accuracy. Certified policies can skip helper messages in 59-80% of cases while retaining 76-95% of performance gains. In other words, most of the chatter between agents is skippable, and a system that can tell which messages matter keeps almost all of the benefit at a fraction of the latency. m:…9773
Replanning without trusting everything
Restraint also has a security dimension. Kaer is a preprint about security-aware replanning for multi-agent systems, and its central idea is to stop trusting feedback just because it arrives. Kaer shields the planner from untrusted agent feedback by conveying diagnostics to the replanning procedure strictly through abstract templates. The test bed is Cintra, a benchmark of simulated user tasks requiring replanning paired with adversarial objectives targeting the replanning phase. Kaer achieves higher robustness than Magentic-One and ControlValve while maintaining comparable utility to ControlValve. The design lesson is simple: the planner should hear what went wrong, not who said so, and the template is what enforces that separation. m:…9778
A fourth preprint tackles the personal agent, where the stakes are different because the agent acts for one person over a long period. Restricting recursive self-improvement to prompt updates while keeping model weights and runtime code fixed allows for effective safety evaluation and repair. Automated repair procedures can significantly reduce security violations and increase safe task completion in personal AI agents, and repair strategies for personal agent instructions are transferable across different model architectures. That last point matters: a fix learned on one model does not have to be relearned on the next one, which is what makes repair a practical control rather than a one-off patch. m:…7398
Why restraint is the product now
These four preprints are about different systems, but they point at the same shift. The first generation of agents was judged on completion: did it finish the task? The next generation will be judged on judgement: did it know when to stop, which messages to ignore, which feedback to distrust, and which of its own instructions needed repair? The numbers suggest that judgement is still the weak point. A 59.5% abstention score is not a passing grade for an agent with irreversible actions, and the fact that abstention is largely independent of task-solving capability means that buying a more capable model does not buy a more careful one. The good news in this week's papers is that restraint can be engineered: skip the useless messages, template the untrusted feedback, repair the risky instructions. The bad news is that none of that is automatic, and the benchmarks suggest most teams have not built it yet. m:…9758 m:…9773
What is still unresolved
- Whether abstention scores improve when agents are given explicit irreversible-action warnings, or whether the failure is deeper.
- Whether certified message-skipping holds up when helper agents are adversarial rather than merely unhelpful.
- How automated repair of personal-agent instructions behaves over months of accumulated updates.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.