The memory tax is the price an agent pays when its past is rewritten before it is searched. In a large October evaluation, dedicated memory systems scored below plain retrieval over raw history in 90 of 99 head-to-head results, according to a synthesis of the study by The AI Operator. The systems did not mainly fail at answering. They failed earlier, at construction, when a fact was dropped, buried or rephrased on its way into the memory store.
The memory tax, measured
He and co-authors at the University of Illinois Urbana-Champaign and partner institutions submitted Memory Tax: Dissecting the Net Value of Agent Memory to arXiv on 25 October 2026. The paper tested 11 dedicated memory systems, including A-Mem, MemOS, MemoryOS, OpenViking, HippoRAG2 and Mem-T, across three benchmarks, LongMemEval-s, PersonaMem-v2 and MINT-Eval, and three model backbones, Qwen3.5-4B, Qwen3.6-27B and GPT-5.6 Luna, as summarised by The AI Operator and by Reid Marlow. The baseline was deliberately crude: one or two rounds of retrieval over raw history chunks.
On LongMemEval-s with Qwen3.6-27B, naive two-round retrieval scored 81.4 percent. A-Mem scored 79.4 percent, LightMem 78.8 percent, MemOS 76.6 percent, SimpleMem 73.6 percent and MemoryOS 73.4 percent, according to Marlow's reading of the paper. On the same benchmark with GPT-5.6 Luna, search scored 90.0 and the best memory tools scored 84.0, according to The AI Operator.
A second team reached the same result from a different corpus. Bao and co-authors submitted Is Complex Agentic Memory Always Needed? to arXiv on 22 October 2026. Their Raw-RAG baseline, dense retrieval plus reranking over raw history, ranked first among the compared memory systems in six of seven benchmark and model settings across LoCoMo, LongMemEval, MEME and MemoryAgentBench, beating the strongest agentic memory system by 0.13 to 5.53 points. A third study, by Zhou and co-authors, compared six explicit memory systems with raw-history retrieval on LongMemEval-S and BEAM-100K and found no consistent advantage for explicit memory, despite construction costs reaching hours per conversation.
Why rewriting loses evidence
The Memory Tax paper names the mechanism. A memory system changes how raw information is organised and accessed. The change can help, by distilling a preference or reorganising a fact. It can also remove or obscure evidence. The authors call the gain memory benefit and the loss memory tax.
The loss is measurable at construction. When the researchers sampled facts, 11.4 percent of MemOS facts had suffered partial or major information loss and another 2.8 percent had no memory unit created at all. For OpenViking, 22.6 percent suffered partial or major loss and 38.0 percent had no memory unit generated, according to Marlow's summary of the sampling. A fact that never enters the store cannot be retrieved, however good the retriever is.
What survives can still be harder to find. Raw chunks keep the original wording, so keyword and dense search can match the exact terms a person used. A memory unit carries the phrasing of the prompt or model that distilled it. Bao and co-authors found that higher coverage of necessary evidence in raw history offset lower utilisation, and that even complete access to a transformed memory store left an accuracy gap against raw evidence. Zhou and co-authors separated the two failure points: some systems fail to preserve answer-supporting information during construction, while others preserve it and then fail to retrieve it effectively.
Where memory still wins, and the caveat
Memory is not useless. It is local. In the Zhou study, memory tools beat search on knowing when to say I do not know, by up to 15 points, and by small margins on putting events in the right order and on sorting out contradictory facts, according to The AI Operator. The Memory Tax authors also found benefit where cross-session synthesis or preference aggregation is the task. The claim the three papers support is narrower than the headline: rewriting history does not pay for itself as a default.
Two caveats belong on the record. First, all three studies are October 2026 preprints, not peer-reviewed results, and they test personal conversation history, not a reviewed company record. Second, vendors report higher scores under their own setups, which do not compare directly, a point The AI Operator notes for Mem0. Neither caveat restores the missing facts. Both argue for measuring a memory system on its own workload, against a raw baseline, before trusting it.
The papers also test a repair. When retrieval runs over raw history alongside the memory store, pooling and reranking both sets of candidates, the tax falls. In one reported test, adding the raw history cut SimpleMem's tax from 12.0 to 5.2 points and raised accuracy from 73.6 to 82.8, as summarised from the Memory Tax results. The original was not a fallback. It was part of the retrieval set.
The source-first test for a company brain
For a company record, the cost of a dropped fact is higher than a forgotten commute time. Extraction can drop an approval condition, a price exception or a safety step, and no later search can recover what construction removed. Three checks follow from the evidence:
- Keep the original as the record. An extracted fact is an index entry that points back to its source, never a replacement for it.
- Make loss visible. Every extracted unit should carry its source, its date and the version it supersedes, so a reviewer can see what changed and what is missing. This is the standard set out in How Jilu Works and in the Context Engineering pillar.
- Beat the raw baseline first. A memory layer should have to outperform plain search over the originals, by question type, on the company's own questions, before its complexity is paid for. The grading problem behind that test is covered in Agent Memory Benchmarks.
That is the distinction Jilu is built on. A draft extracted from a source waits for a person to review and publish it, and the source stays attached to the record, so a correction reaches the original and the extracted fact at once. The October evidence does not say companies should skip structure. It says structure that discards its source pays a tax, and the tax is largest on the facts nobody noticed leaving.