Agent memory only pays when the evidence stays checkable

When memory clearly wins

LangChain describes an experiment in which an agent using OpenWiki as a memory layer was 5x more accurate than one using grep tools alone on questions spanning multiple sources. The gap is large because the task requires combining evidence from several documents, where the relationships between fragments decide the answer. m:…1864

Adding an evidence graph to the wiki improved accuracy by 27% and doubled citation consistency, and the evidence graph cut cost per correct answer by 58% compared with the wiki alone. Accuracy, citations and cost usually trade against each other. Here the graph improved all three at once. m:…1864

The mechanism is concrete rather than mystical. The evidence graph uses typed edges to represent relationships like refers_to and supersedes, and OpenWiki uses hashes of source lines to flag affected claims without requiring a model call. Graph construction took under a second. That is what checkable means in practice: the memory knows its own provenance. m:…1864

The test behind these numbers was deliberately narrow, and LangChain reports its shape honestly. The test library consisted of 111 synthetic insurance documents, and the test ledger contained 972 facts used to score agent accuracy. Compiling the 111-document library into a wiki cost $4.15, and adding documents to an existing OpenWiki library costs approximately $0.09 per document. Those costs are small, but the synthetic setting is a limit. Insurance documents with a clean fact ledger are friendlier ground than messy real-world corpora, and teams should expect noisier results outside it. m:…1864

There is also a boundary on when memory helps at all. For simple lookups, agents without memory were just as accurate and faster than agents with memory. Memory is overhead when the answer lives in one place and the question does not require synthesis. The design question is therefore not whether to add memory, but which questions deserve it. m:…1864

When memory becomes a tax

A preprint with a deliberately provocative name, Memory Tax, argues the other side of the same question. It found dedicated memory systems often underperform naive retrieval-augmented generation, and many dedicated memory systems underperform naive RAG even at higher costs. The authors conducted a large-scale evaluation across multiple benchmarks and model backbones. The blunt version is that adding a memory system can make an agent worse and more expensive at the same time. m:…1511

The preprint gives a mechanism for the failure. Memory tax arises because memory processes change how raw information is organized and accessed, and memory processes may remove or obscure evidence. Summarizing or restructuring a source can destroy exactly the detail a later question needs. Memory benefit reflects the expected gains of memory design; the tax is the price when it guesses wrong about what to keep. m:…1511

Read against the OpenWiki results, the two studies agree more than they disagree. Memory wins when its structure preserves evidence and its links, and it loses when its processing obscures the source. The preprint authors say they will open-source the code and results to the community, which lets teams test that distinction on their own data. m:…1864 m:…1511

Learning when to remember

A second preprint tackles the question OpenWiki leaves open: even with good memory, when should an agent use it? DiscernMem improves memory quality by comparing team executions with and without memory to diagnose rule defects. Boundary-Learning Meta-Memory reduces inappropriate memory usage by learning when a role should apply or avoid specific memories based on task context. m:…5914

DiscernMem achieved superior performance on AutoGen, CAMEL and MacNet benchmarks compared with existing baselines, with a PDDL score of 37.71 versus 28.48 on CAMEL. Knowing when not to use a memory may matter as much as building the memory in the first place. m:…5914

EMMA, a preprint on structured episodic memory for multi-agent workflow synthesis, attains the best overall performance on KramaBench, AppWorld and BBEH, outperforming ACE, GEPA, EvoMAS and ADAS by up to 32.5%, while using 17 to 38% fewer training executions than the most efficient baseline. Backbone-independent guards and a reflector mechanism enable scoped, incremental updates to multi-agent configurations, preventing cross-task interference. m:…0551

The hardest test may be habits, where evidence is sparse by definition. HABIT-Bench evaluates whether LLM agent memory systems can infer implicit user habits from sparse dialogue evidence, using synthetic user histories spanning hundreds of sessions across four domains: food, finance, software and travel. The best-performing memory method achieved 33.61% macro accuracy, and performance remains limited even when annotated evidence is supplied directly. Random guessing achieves 25% accuracy on the four-choice tasks, so the best system is only modestly above chance. Tasks include temporary exceptions, changes in habits and suggestions users have not accepted, all of which punish a memory that treats the past as a fixed rule. m:…6173

A rule for builders

The practical rule that survives all these studies is simple. Build memory that preserves a path back to the source, measure it against plain retrieval on your own questions, and gate its use by task type. If the memory cannot show its evidence, or does not beat retrieval on the questions you actually ask, it is overhead. If it can show its evidence and wins on multi-source questions while staying out of the way on simple ones, it earns its cost. m:…1864 m:…1511

That rule also explains why the OpenWiki cost numbers matter. A memory layer that costs $4.15 to compile for 111 documents and about $0.09 per added document is cheap to test and cheap to abandon. The expensive mistake is committing a whole agent fleet to a memory design before checking whether it beats retrieval. m:…1864