Long-horizon agents are now a memory problem, not a reasoning problem

The KV bill is the agent bill

In a preprint analyzing KV residency in production-scale agentic LLM workloads, Microsoft's authors quantify what serving teams have felt: the memory ceiling is the agent bill. A one-hour idle cap on KV state retention reduces the total HBM memory ceiling by 3.6x while increasing recomputation rates by only 0.46%. Under tight memory budgets, multi-resolution retention — demoting KV state to int4 — reduces recomputation rates to 13.6%, compared to 63% for LRU eviction. Demoting beats evicting, by a wide margin. m:…7879

The catch is routing. When requests are routed across different deployments, cached-context reuse drops from 89% to 29%. The cache you budgeted for evaporates with the routing policy — which means agentic memory pressure is as much a placement problem as a capacity problem. m:…7879

Memory rebuilt as task objects

The research answer starts with what memory is for. CarryOn, in a preprint, argues that long-horizon agents should rebuild context around reusable task objects instead of raw interaction histories: "CarryOn organizes long-horizon agent context around reusable task objects rather than interaction histories". The results: improved gains of 5.8% on OSWorld 2.0 and 7.0% on BrowseComp-Plus, while total token usage falls 51.9% on OSWorld 2.0. Less context, better organized, better scores. m:…8530

The idea is object-oriented programming applied to agent memory: persistent task objects that agents carry across steps, rather than an ever-growing transcript. The 51.9% token cut is the number that should catch an infrastructure team's eye — it is compute savings and memory savings at once. Mechanically, CarryOn builds a context view from continuation carriers, complementary execution context, and a live tail of recent interactions — carrying forward unaffected records and jointly updating execution context rather than replaying the transcript. On BrowseComp-Plus it cut token usage 22.9% alongside the 7.0% gain. m:…8530

The design keeps the object model honest: CarryOn maintains carrier records across folds, revising affected records against bound evidence and applicable environment checks. When the environment contradicts a record, the record gets revised — the agent does not just accumulate assertions. That is the difference between a memory and a log. m:…8530

Choosing what to keep

JointCore, in another preprint, attacks the selection problem instead of the storage problem. Rather than scoring individual memory items by marginal influence, it tests blocks of records and recovers a minimum-cost sufficient set in 100% of 6,912 exact conditions. That block-wise approach reduces subset evaluations from 30.00 to 11.13 — a 62.9% reduction — and achieves 87.8% active-memory savings across 12B to 70B model families. m:…9477

The lesson generalizes: memory selection is a joint search problem, not a per-item ranking problem. Testing items in interaction-aware blocks finds the sufficient set that marginal scoring misses — and the savings scale across model sizes, not just one benchmark sweet spot. m:…9477

Build the memory layer first

Put the three threads together and the picture is a deployment checklist. Expect agentic KV pressure to set your memory budget, not your model weights. Demote rather than evict when budgets tighten. Keep related requests on the same deployment so the cache survives routing. And rebuild agent memory around task objects and block-wise selection instead of raw histories. All three results are preprints, so treat the exact percentages as the authors' measurements rather than law — but the direction of travel is unambiguous: the long-horizon agent is a memory system first and a reasoning system second. m:…7879 m:…8530