Every agent that remembers a past conversation is paying a hidden toll. Most memory systems call the underlying language model twice over: once to compress what happened into a stored note, and again to search those notes when a new question arrives. A preprint posted to arXiv argues that toll is unnecessary, and describes a system, Zero-Mem, that removes the language model from both steps entirely.
The problem the authors describe is structural. Long-running agents, coding assistants, customer support bots, personal assistants that persist across weeks, accumulate interaction history that cannot simply be replayed in full at every turn. The common fix is to have the model itself write summaries or structured records of what occurred, then have the model again decide which records are relevant to a new query. Each of those steps burns tokens and adds latency, on top of whatever the agent spends generating its actual response. The authors also note a subtler cost: summarizing throws away detail, so the record the agent later retrieves may be a blurred version of what really happened.
Zero-Mem’s fix is to stop generating anything at all during memory operations. Instead of writing model-generated summaries, it keeps the raw interaction transcript as its permanent record and organizes that transcript two different ways. One structure is a graph connecting entities and context across separate interactions, useful for surfacing a fact mentioned sessions ago. The other is a timeline that preserves the order of a conversation and which session each exchange belonged to, useful for questions that depend on recency or sequence. When a new query comes in, the system checks both structures, weighs which one is more useful for that particular question, and pulls supporting material from whichever (or both) apply. A filtering step then discards evidence that contradicts itself before handing the retrieved material to a language model for the first and only time: to write the final answer.
According to the authors, this design holds its accuracy against rival systems on the standard tests for answering questions over long conversational histories and very large contexts, while using zero language-model calls and zero language-model tokens anywhere in the memory pipeline itself. Holding the same answering model and context size constant, they report a 57.6 percent reduction in the time memory operations take relative to the fastest system they compared it against. They also ran ablation tests removing each of the two organizing structures separately, which they say confirms both contribute and that the system’s choice between them, rather than either one alone, drives the result.
This is a preprint, submitted July 31, 2026, and the results are the authors’ own; it has not gone through peer review, and the authors say code will be released only after that review completes. The claim of matching, not beating, existing systems on accuracy is also worth sitting with: the win here is cost, not capability, and the 57.6 percent time figure is measured against one named baseline rather than the full field.
Memory has become the quiet line item in agent economics. Model calls for reasoning and generation get budgeted and benchmarked; the calls a memory layer makes in the background rarely do, even though they scale with every turn an agent takes over a long session. Teams operating agents with persistent memory should audit how many of their own memory-layer calls actually touch a language model, and treat that count as a cost line worth optimizing on its own, independent of how good the agent’s answers are.
Preprint published on arXiv on July 31, 2026.
