Agent training gets practical: environments, optimizers, and programmatic tools

The environment is the product

Invisible Technologies, writing from enterprise deployment work, argues the most common failure mode in enterprise AI deployment is not a bad model but a training environment that does not reflect the conditions where the model will operate. Well-designed RL environments close that gap with verifiers that produce rubrics, structured criteria scoring outputs across multiple dimensions rather than a single pass or fail signal. m:…3080

The post makes two practical claims. First, reinforcement learning with verifiable rewards uses programmatic verifiers that scale in ways human-rated RLHF cannot, which is what lets training iterations complete in days rather than months. Second, the binding constraint is not infrastructure but expertise: verifier quality matters more than rollout volume in the early stages, reliable verifiers are the rate-limiting factor, and most projects stall because the team cannot write rubrics that accurately reflect production quality. In other words, the scarce input is domain experts, not GPUs. m:…3080

Sharper recipes for long horizons

A preprint posted this week takes on the training loop itself for long-horizon agents, which must coordinate reasoning, tool use, and environment feedback over extended interactions. The authors study a single-rollout asynchronous recipe that updates the policy while rollouts are still being collected, combining the Muon optimizer with token-averaged loss, truncated importance sampling, and auxiliary penalties on response length and tool errors. m:…1460

Evaluated on repository-level issue resolution across two model families, the full recipe achieved higher training success than the baselines. Single-rollout sampling matters because it removes the need to wait for multiple trajectories on the same issue before learning, which is what makes the recipe asynchronous. The paper reads as a direct answer to the environment problem: once you have a good verifier, the question becomes how efficiently you can learn from it. m:…1460

Programmatic tool calling

A second preprint, ProgAgent, argues the interface between the agent and its tools needs retraining too. Most agents use direct tool calling, invoking tools sequentially while tool responses accumulate in the model context, which the authors say leads to context rot and orchestration breakdown on long-horizon tasks. Programmatic tool calling instead has the model invoke tools through executable code and store intermediate results in runtime variables rather than appending them to context. m:…5431

The results are striking for how little data they required: using only 215 tasks instantiated at four environment scales, ProgAgent-14B reached a 30.7% pass rate on LOCA-bench at a 64k-token environment state, where the evaluated direct-tool-calling baselines scored zero. Cross-harness experiments showed the programmatic-tool-calling models outperforming direct-tool-calling models under both evaluation interfaces, suggesting the benefit is in the training, not the harness. m:…5431

Training collaboration itself

A third preprint, CoSIR, extends the training agenda to multi-agent collaboration. Rather than only tuning an inference-time workflow, the method post-trains the underlying LLM with self-play, using a Bayesian evaluator that scores shared information by how much it reduces the team's surprisal, a fine-grained reward for telling teammates what they need and using what they are told. m:…9209

CoSIR achieved the highest aggregate score across seven unseen collaboration tasks on three backbones, with Qwen3-8B improving 31.85 points over its base model. Private-fact disclosure rose from 68.5% to 74.3%, and failures to use teammates' input fell from 26.3% to 22.8%, while GSM8K, MMLU, and IFEval performance stayed largely preserved. This suggests collaboration, like tool use, is a capability that can be trained rather than merely prompted. m:…9209