Agents are now rewriting their own instructions. The new rule: prove it, or it does not ship.

The agent that writes its own playbook

In a preprint this week, researchers describe ACE-RL, a reinforcement learning method that enables language-model agents to improve by creating and curating experience artifacts. It trains one model to act, reflect, and curate task-local playbooks within a fixed harness, co-optimizing the action generator, the insight reflector and the playbook curator. The harness allows models to reflect on attempts and write notes in a playbook, and ACE-RL evaluates playbook edits by their downstream effect on future actor rollouts, so only edits that help later attempts survive. The method achieves higher held-out performance than baselines including PPO, GRPO and ERL, and its gains persist beyond the training cutoff, maintaining an advantage over GRPO up to 8 attempts. m:…2987

The detail that matters is what does not work. Language-model agents trained with reinforcement learning often discard fine-grained environment information, and training only the actor is insufficient for optimal agent performance in the harness. A GRPO-trained model in the ACE harness after 8 attempts performs worse than simple sampling. The lesson is that the reflection and curation roles are not decoration: each role is compared only against other outputs of the same role to keep the learning signal stable. An agent that only acts gets worse with practice. An agent that writes down what it learned gets better. m:…2987

The agent that rebuilds its own harness

A second preprint goes further: the agent as its own optimizer. It presents a framework for self-evolving agent harnesses, where a frozen model iteratively improves its own execution environment by analyzing run records, optimizing its own tool use, context management and prompt organization. Agents acting as their own optimizers can improve harness performance by approximately 5 points on in-distribution benchmarks, and multi-task pretraining and continual training enable generalization to out-of-distribution benchmarks. The evolved harnesses exhibit specific mechanisms including output truncation, history compaction and completion checks. Nobody designed those mechanisms for the task; the agent discovered that they help, and kept them. m:…6582

A third preprint, Agent-R2, attacks the data side of the same problem. It is a real-source data synthesis framework for training agents on realistic productivity work, distilling 4k task instances from 10k web-corpus candidates, grounded in real occupational seeds and authentic multi-format files. The resulting 4B-parameter model achieves a 51.0 percent pass rate on the GDPval benchmark and 14.8 percent on OfficeQA, ahead of the 235B-parameter Qwen3-235B-A22B-Thinking-2507 at 48.0 and 13.8 percent. Removing the rubric reward drops reinforcement learning performance from 51.0 to 49.5 percent on GDPval, and real files outperform synthetic ones in ablation studies, 43.3 versus 41.4 percent. The framework also mitigates shortcut behaviour in agents, reaching a 90.9 percent completion rate. A small model trained on real work beats a giant one trained on synthetic work, which is a cheaper lesson than it sounds. m:…2593

The governor

The obvious worry is an agent that rewrites itself into something worse, and a fourth preprint is the answer to that worry. Evidence-Governed Evolution separates candidate generation from confirmatory adoption to prevent degradation against earlier baselines, and uses reference-relative non-inferiority tests to ensure new agent versions do not regress on established capabilities. Its safety guarantees are conditional on bounded paired scores, fresh sampling and valid reference-evaluation constructs, which is the authors' way of saying the statistics only protect you if the evaluation is honest. The framework provides a statistical basis for audit planning, relating effect-size margins and sample sizes to acceptance probability. Self-improvement is allowed; self-degradation is not, and the difference is a test the change has to pass. m:…9305

Taken together, the four preprints describe a loop with a governor: agents that reflect and curate their own playbooks, rebuild their own harnesses, train on real work rather than synthetic work, and adopt self-generated changes only when the evidence clears a statistical bar. All four are preprints, and their numbers are the authors' own benchmark results, so treat the point gains and pass rates as claims to verify, not facts. But the direction is hard to miss. The next generation of agents will not just run the workflow you wrote. They will rewrite it, and the teams that deploy them will need the evidence gate more than the agent does. m:…2987 m:…6582 m:…9305 m:…2593