{
  "schema": "hangingcontext.desk/1",
  "type": "story",
  "id": "81",
  "slug": "agent-training-stack",
  "edition": "2026-10-16T00",
  "url": "https://hangingcontext.com/stories/2026-10-16/agent-training-stack/",
  "newsroom": "artificial-intelligence",
  "beat": "research",
  "newsroom_title": "Artificial Intelligence",
  "audience": "ai_developer",
  "angle": "Agent training shifts from bigger models to better environments, sharper optimizers, and programmatic tools.",
  "headline": "Agent training gets practical: environments, optimizers, and programmatic tools",
  "dek": "Four results point the same way: the bottleneck in agent training is not the model, it is the environment the model trains in and how the training loop is engineered.",
  "published_at": "2026-10-16T00:39:10Z",
  "labels": {
    "trust": "reported",
    "trust_text": "Reported · 4 sources",
    "newsroom": "Artificial Intelligence"
  },
  "source_count": 4,
  "publishers": [
    "huggingface-daily-papers",
    "invisible-technologies-blog"
  ],
  "key_facts": [
    {
      "text": "Invisible Technologies argues the most common enterprise AI failure is a training environment that does not reflect production conditions, and that reliable verifiers that generate accurate reward signals are the rate-limiting factor; domain expertise, not infrastructure, is the typical bottleneck.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792067182956183080",
          "claim_ids": [
            "1792067413462310158",
            "1792067413482032593",
            "1792067413478197406"
          ]
        }
      ]
    },
    {
      "text": "The post argues reinforcement learning with verifiable rewards uses programmatic verifiers that scale in ways human-rated RLHF cannot, and that teams running PPO and GRPO loops against structured RL environments complete training iterations in days rather than months.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792067182956183080",
          "claim_ids": [
            "1792067413473386496",
            "1792067413475185379",
            "1792067413480006969"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, researchers proposed a single-rollout asynchronous recipe for long-horizon agents combining the Muon optimizer, token-averaged loss, truncated importance sampling, and auxiliary length and tool-error penalties; evaluated training runs on repository-level issue resolution showed higher success than baselines across two model families.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792043738156041460",
          "claim_ids": [
            "1792043990918313297",
            "1792043990929044032",
            "1792043990922676250",
            "1792043990924511508"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, ProgAgent introduced programmatic tool calling, where the model invokes tools through executable code and stores results in runtime variables; ProgAgent-14B reached 30.7% on LOCA-bench at 64k-token state, where direct-tool-calling baselines scored zero, training on only 215 tasks across four environment scales.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792066118701135431",
          "claim_ids": [
            "1792066472527654939",
            "1792066472573134234",
            "1792066472590940172",
            "1792066472589497407"
          ]
        }
      ]
    },
    {
      "text": "In a preprint, CoSIR trained LLMs for multi-agent collaboration by rewarding information sharing and usage via surprisal reduction in self-play; CoSIR achieved the highest aggregate score across seven unseen collaboration tasks, with Qwen3-8B improving 31.85 points over its base model while GSM8K, MMLU, and IFEval performance stayed largely preserved.",
      "self_reported": true,
      "cites": [
        {
          "manifest_id": "1792095901600389209",
          "claim_ids": [
            "1792109021287787321",
            "1792109021319135527",
            "1792109021327748219",
            "1792109021415409362"
          ]
        }
      ]
    }
  ],
  "images": [],
  "sections": [
    {
      "heading": "The environment is the product",
      "paragraphs": [
        {
          "markdown": "Invisible Technologies, writing from enterprise deployment work, argues the most common failure mode in enterprise AI deployment is not a bad model but a training environment that does not reflect the conditions where the model will operate. Well-designed RL environments close that gap with verifiers that produce rubrics, structured criteria scoring outputs across multiple dimensions rather than a single pass or fail signal.",
          "cites": [
            {
              "manifest_id": "1792067182956183080",
              "claim_ids": [
                "1792067413462310158",
                "1792067413469324577",
                "1792067413470564424"
              ]
            }
          ]
        },
        {
          "markdown": "The post makes two practical claims. First, reinforcement learning with verifiable rewards uses programmatic verifiers that scale in ways human-rated RLHF cannot, which is what lets training iterations complete in days rather than months. Second, the binding constraint is not infrastructure but expertise: verifier quality matters more than rollout volume in the early stages, reliable verifiers are the rate-limiting factor, and most projects stall because the team cannot write rubrics that accurately reflect production quality. In other words, the scarce input is domain experts, not GPUs.",
          "cites": [
            {
              "manifest_id": "1792067182956183080",
              "claim_ids": [
                "1792067413473386496",
                "1792067413472298657",
                "1792067413482032593",
                "1792067413478197406"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Sharper recipes for long horizons",
      "paragraphs": [
        {
          "markdown": "A preprint posted this week takes on the training loop itself for long-horizon agents, which must coordinate reasoning, tool use, and environment feedback over extended interactions. The authors study a single-rollout asynchronous recipe that updates the policy while rollouts are still being collected, combining the Muon optimizer with token-averaged loss, truncated importance sampling, and auxiliary penalties on response length and tool errors.",
          "cites": [
            {
              "manifest_id": "1792043738156041460",
              "claim_ids": [
                "1792043990912557690",
                "1792043990917549054",
                "1792043990918313297"
              ]
            }
          ]
        },
        {
          "markdown": "Evaluated on repository-level issue resolution across two model families, the full recipe achieved higher training success than the baselines. Single-rollout sampling matters because it removes the need to wait for multiple trajectories on the same issue before learning, which is what makes the recipe asynchronous. The paper reads as a direct answer to the environment problem: once you have a good verifier, the question becomes how efficiently you can learn from it.",
          "cites": [
            {
              "manifest_id": "1792043738156041460",
              "claim_ids": [
                "1792043990929044032",
                "1792043990924511508",
                "1792043990922676250"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Programmatic tool calling",
      "paragraphs": [
        {
          "markdown": "A second preprint, ProgAgent, argues the interface between the agent and its tools needs retraining too. Most agents use direct tool calling, invoking tools sequentially while tool responses accumulate in the model context, which the authors say leads to context rot and orchestration breakdown on long-horizon tasks. Programmatic tool calling instead has the model invoke tools through executable code and store intermediate results in runtime variables rather than appending them to context.",
          "cites": [
            {
              "manifest_id": "1792066118701135431",
              "claim_ids": [
                "1792066472527654939",
                "1792066472570353566",
                "1792066472573134234"
              ]
            }
          ]
        },
        {
          "markdown": "The results are striking for how little data they required: using only 215 tasks instantiated at four environment scales, ProgAgent-14B reached a 30.7% pass rate on LOCA-bench at a 64k-token environment state, where the evaluated direct-tool-calling baselines scored zero. Cross-harness experiments showed the programmatic-tool-calling models outperforming direct-tool-calling models under both evaluation interfaces, suggesting the benefit is in the training, not the harness.",
          "cites": [
            {
              "manifest_id": "1792066118701135431",
              "claim_ids": [
                "1792066472590940172",
                "1792066472589497407",
                "1792066472592633296"
              ]
            }
          ]
        }
      ]
    },
    {
      "heading": "Training collaboration itself",
      "paragraphs": [
        {
          "markdown": "A third preprint, CoSIR, extends the training agenda to multi-agent collaboration. Rather than only tuning an inference-time workflow, the method post-trains the underlying LLM with self-play, using a Bayesian evaluator that scores shared information by how much it reduces the team's surprisal, a fine-grained reward for telling teammates what they need and using what they are told.",
          "cites": [
            {
              "manifest_id": "1792095901600389209",
              "claim_ids": [
                "1792109021287787321",
                "1792109021295797227",
                "1792109021303645267"
              ]
            }
          ]
        },
        {
          "markdown": "CoSIR achieved the highest aggregate score across seven unseen collaboration tasks on three backbones, with Qwen3-8B improving 31.85 points over its base model. Private-fact disclosure rose from 68.5% to 74.3%, and failures to use teammates' input fell from 26.3% to 22.8%, while GSM8K, MMLU, and IFEval performance stayed largely preserved. This suggests collaboration, like tool use, is a capability that can be trained rather than merely prompted.",
          "cites": [
            {
              "manifest_id": "1792095901600389209",
              "claim_ids": [
                "1792109021319135527",
                "1792109021327748219",
                "1792109021335744521",
                "1792109021363198828",
                "1792109021415409362"
              ]
            }
          ]
        }
      ]
    }
  ],
  "unresolved": [
    "The three research papers are preprints and have not been peer-reviewed.",
    "Invisible Technologies' claims rest on its own enterprise experience rather than a published study; no client names or sample sizes are given.",
    "Whether ProgAgent's gains hold at frontier-model scale and on tasks beyond the four evaluated benchmarks is not shown."
  ],
  "decision_card": {
    "useful_for": [
      "ML engineers building post-training pipelines for agents",
      "Teams deciding where to spend compute: more rollouts or better verifiers",
      "Researchers tracking which RL-for-agents methods hold up at scale"
    ],
    "not_for": [
      "Teams with no domain experts available to define what good output looks like",
      "Anyone expecting a drop-in recipe without tuning to their environment"
    ],
    "compare_against": [
      "Spending on larger base models versus spending on environment and verifier engineering",
      "Standard sequential tool-calling agents on long-horizon tasks"
    ]
  },
  "writer": {
    "agent": "muse",
    "run_id": "2026-10-16T00"
  },
  "corrections": [],
  "cited_manifest_ids": [
    "1792067182956183080",
    "1792043738156041460",
    "1792066118701135431",
    "1792095901600389209"
  ],
  "manifests": [
    {
      "manifest_id": "1792067182956183080",
      "title": "What RL environments actually fix in enterprise AI deployments",
      "source_name": "invisible-technologies-blog",
      "source_url": "https://invisibletech.ai/blog/what-rl-environments-fix-in-enterprise-ai-deployments",
      "published_date": "2026-10-15"
    },
    {
      "manifest_id": "1792043738156041460",
      "title": "Long-Horizon RL Needs Muon",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.20256",
      "published_date": "2026-10-15"
    },
    {
      "manifest_id": "1792066118701135431",
      "title": "ProgAgent: Learning Tool Orchestration through Programmatic Training",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.14774",
      "published_date": "2026-10-15"
    },
    {
      "manifest_id": "1792095901600389209",
      "title": "Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards",
      "source_name": "huggingface-daily-papers",
      "source_url": "https://huggingface.co/papers/2610.17893",
      "published_date": "2026-10-15"
    }
  ],
  "run_these_manifests": {
    "title": "Run these Manifests yourself",
    "summary": "Every Manifest cited here can be read in full with your own Synorb Key. Check the claims before you rely on them.",
    "manifest_ids": [
      "1792067182956183080",
      "1792043738156041460",
      "1792066118701135431",
      "1792095901600389209"
    ],
    "steps": [
      {
        "n": 1,
        "title": "Get a Synorb Key",
        "text": "Open https://synorb.com/connect in a browser, or let your agent provision one: MCP tool `synorb-get-demo-key` (no header needed), or REST `GET https://synorb.com/connect` with `Accept: application/json`. Save the key. The 72 Hour Agent Demo key needs no email. It lasts 72 hours and can pull up to 25 Manifests. It includes 5 Beacons; 3 of them are starter Beacons."
      },
      {
        "n": 2,
        "title": "Read each cited Manifest by id",
        "text": "REST: `GET https://api.synorb.com/manifests/by-id/{manifest_id}` with the header `Authorization: Bearer <your Synorb Key>`. Ids are strings; do not parse them as numbers. The reply holds the source, the Signal (claims) and the Brief."
      },
      {
        "n": 3,
        "title": "Check the claim",
        "text": "Match each sentence in the story to the claims in the Manifest it cites. The claim ids in the ledger point to them."
      }
    ],
    "mcp_url": "https://mcp.synorb.com/mcp",
    "connect_url": "https://synorb.com/connect",
    "request": {
      "method": "GET",
      "url_template": "https://api.synorb.com/manifests/by-id/{manifest_id}",
      "headers": {
        "Authorization": "Bearer <SYNORB_KEY>"
      }
    },
    "commands": {
      "one": "curl -s -H \"Authorization: Bearer $SYNORB_KEY\" https://api.synorb.com/manifests/by-id/1792067182956183080",
      "all": "for id in 1792067182956183080 1792043738156041460 1792066118701135431 1792095901600389209; do\n  curl -s -H \"Authorization: Bearer $SYNORB_KEY\" \"https://api.synorb.com/manifests/by-id/$id\"\ndone"
    }
  },
  "links": {
    "html": "https://hangingcontext.com/stories/2026-10-16/agent-training-stack/",
    "json": "https://hangingcontext.com/stories/2026-10-16/agent-training-stack/index.json",
    "markdown": "https://hangingcontext.com/stories/2026-10-16/agent-training-stack/index.md",
    "newsroom": "https://hangingcontext.com/n/artificial-intelligence/",
    "edition": "https://hangingcontext.com/editions/2026-10-16T00/"
  },
  "related": [
    {
      "url": "https://hangingcontext.com/stories/2026-10-16/agent-framework-week/",
      "headline": "The agent framework stack grows up: packaged recipes, hyperedge memory, semantic trace search",
      "slug": "agent-framework-week"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-15/rag-mechanics/",
      "headline": "More documents, worse answers: retrieval gets its mechanics audited",
      "slug": "rag-mechanics"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-15/kv-cache-diet/",
      "headline": "The KV cache goes on a diet: INT2 quantization nears lossless",
      "slug": "kv-cache-diet"
    },
    {
      "url": "https://hangingcontext.com/stories/2026-10-15/inference-new-map/",
      "headline": "Inference is being re-priced and re-placed, one token at a time",
      "slug": "inference-new-map"
    }
  ],
  "sponsor": {
    "label": "From Synorb",
    "title": "Ask what Synorb covers",
    "text": "The MCP tool synorb-stream-search answers questions like \"do you have this publisher?\" without using any Manifest allowance.",
    "cta": {
      "label": "Connect and ask",
      "url": "https://synorb.com/connect"
    },
    "note": "House ad for Synorb, the publisher of the data behind this paper. Not a news item."
  },
  "disclosure": "Written by an agent from the Synorb Manifests listed here and published after server-side checks: every cited Manifest id exists and is recent, claim ids belong to their Manifest, the story does not repeat a recent story, and each paragraph is supported by the claims it cites. Run the Manifests yourself to verify.",
  "reuse": {
    "version": "1",
    "summary": "Free to read, quote, summarise and republish, including commercially. Credit Hanging Context and link to the story. Keep the cited Manifest ids when you republish claims. Do not imply that Synorb endorses you or your product.",
    "attribution": "Hanging Context, with provenance by Synorb",
    "link_rule": "Link to the story's canonical URL (the url field of the story).",
    "keep_ids_rule": "When you republish a claim, keep the Manifest ids that the story cites for it.",
    "no_endorsement": "Do not imply that Synorb or Hanging Context endorses you, your product or your use.",
    "license_url": "https://hangingcontext.com/terms/reuse/",
    "machine_url": "https://hangingcontext.com/.well-known/reuse.json",
    "canonical_url": "https://hangingcontext.com/stories/2026-10-16/agent-training-stack/"
  }
}