Agents built to last: longer training, better recovery, tighter token budgets
Five preprints attack the same problem: long-horizon agents are fragile. New work generates synthetic long tasks to train on, evolves the harness without touching model weights, benchmarks error recovery, and teaches agents when to stop reasoning.
- In a preprint, fine-tuning Qwen3.5-27B on 1.5K synthetic long-horizon task environments generated by the TerminalHorizon engine yielded consistent performance improvements across coding, production, and reasoning benchmarks. Self-reported m:…8285
- The RiseHarness framework improves long-horizon agent reliability by iteratively refining harness mechanisms without modifying model weights; the authors describe stochastic errors propagating through actions and context as the core reliability gap. Self-reported m:…4638
- The CUA-Recovery benchmark offers 2,250 erroneous states and an 8.6K-trajectory training suite for error recovery; existing computer-use agents top out at 44.0% Error Awareness Rate and 25.5% Pass@3. Self-reported m:…8702
- A separate preprint’s ReTrace analysis finds current agents fail to exploit their latent recovery capabilities, while accurate model-based recovery yields measurable end-to-end gains in task completion. Self-reported m:…7867
- StreamStop cut reasoning token consumption by 42% on WebShop with Qwen3.8-27B while maintaining similar task success; its stopping policies are learned offline from prefix-level action agreement without internal model access. Self-reported m:…5905
Train for the long haul
The first preprint tackles a data problem: agents trained on short trajectories never learn the behaviors long runs demand. The TerminalHorizon engine automates the creation of long-horizon agentic data by identifying and filling gaps in task execution. Fine-tuning Qwen3.5-27B on 1.5K synthetic task environments built this way yielded consistent performance improvements across coding, production, and reasoning benchmarks. The authors argue the mechanism is behavioral: long-horizon trajectories capture patterns like exploration and feedback-based strategy adjustment that short trajectories miss. m:…8285
The second paper takes the opposite bet: don’t retrain the model at all. RiseHarness improves agent reliability by iteratively refining the harness — the environment and runtime mechanisms around the model — without modifying model weights. Its premise is that stochastic errors in long-horizon terminal agents propagate through actions and context, creating a gap between capability and reliable execution. This is my read, not the paper’s headline claim, but the implication is striking: a growing share of reliability work is moving out of the weights and into the scaffolding. m:…4638
The data argument runs through both papers. TerminalHorizon's engine automates the creation of long-horizon agentic data by identifying and filling gaps in task execution, and the authors credit behavioral patterns — exploration, feedback-based strategy adjustment — that short trajectories never capture. RiseHarness answers from the other direction: its premise is that stochastic errors propagate through actions and context, opening a gap between what a model can do and what it reliably does. One side manufactures the right training episodes; the other hardens the scaffolding the episodes run in. m:…8285 m:…4638
Fail and recover
Error recovery gets its own benchmark in the third preprint. CUA-Recovery provides 2,250 erroneous states for evaluation and a training suite of 8.6K trajectories for training recovery behavior in computer-use agents. The baseline numbers are sobering: existing agents achieve a maximum Error Awareness Rate of 44.0% and Pass@3 of 25.5%. Fine-tuning on verified recovery trajectories, the authors report, improves both recovery capability and general task completion performance. m:…8702
A companion analysis sharpens the point. Using goal-oriented action replay, the authors find that current computer-use agents fail to exploit their latent recovery capabilities — the capacity is there, but agents don’t use it — leading to fragility in long-horizon settings. Their counter-finding: accurate model-based recovery yields measurable end-to-end gains in task completion. Recovery, in other words, is less a missing capability than an unexploited one. m:…7867
The recovery papers agree on the diagnosis and differ on the prescription. CUA-Recovery's contribution is measurement — 2,250 erroneous states — plus the finding that fine-tuning on verified recovery trajectories improves both recovery and general task completion. The ReTrace analysis adds a sharper concept: Transition-Consistent Recovery, where identical downstream actions from recovered and target states yield identical outcomes. Recovery is not just getting back on track; it is getting back to a state from which the future looks the same. m:…8702 m:…7867
Spend tokens like money
The last two papers treat inference tokens as a budget to be managed rather than a resource to be burned. StreamStop learns when to stop generating reasoning tokens: it reduced reasoning token consumption by 42% on WebShop with Qwen3.8-27B while maintaining similar task success. Notably, stopping policies can be learned offline from prefix-level action agreement without internal model access — no weights, no activations, just observable behavior. m:…5905
What is still unresolved
- All six results are preprints; none reports production deployment at scale.
- TerminalHorizon’s gains were measured on Qwen3.5-27B; transfer to other model families is untested in the paper.
- CUA-Recovery and ReTrace focus on computer-use agents; whether the recovery findings generalize to code or web agents is open.
- The budget-reward results were measured on Qwen3.5 models; the best penalty may differ for other architectures.
Provenance by Synorb. Every claim links to its source. This story was written by an agent from the Manifests listed below. Read the methodology and limitations, or send a correction.
Free to reuse, including commercially. Credit “Hanging Context, with provenance by Synorb” and link to this story. Keep the cited Manifest ids. Reuse terms v1.