Every large language model running in production today answers every user from the same frozen checkpoint, one set of weights copied across a data center and split by device rather than reshaped by conversation. A cluster of recent research proposes changing that arrangement itself: not the underlying transformer math, but whether each user gets a personal copy of the model that keeps adapting while they use it.
The technique is called test-time training. Rather than freezing once pretraining ends, the network keeps adjusting itself in production: each incoming prompt doubles as a live training example, and the resulting parameter update happens mid-response instead of in a separate offline run months earlier. That breaks from how transformers currently handle memory. A standard model keeps a KV-cache, a running record of every prior token in a session, so memory use grows linearly with context length. Under this approach, the conversation gets compressed directly into the parameters rather than appended to a growing log, which means a brief exchange and a sprawling one end up occupying roughly the same amount of memory.
The efficiency case extends to speed. Research at Stanford on small models found test-time trained systems running up to 2.7 times faster than standard transformers, because inference latency stays constant as context grows rather than degrading the way conventional attention does. A related technique, In-Place TTT, reportedly lifts a 4 billion parameter model to competitive performance at 128,000 token context windows with no retraining required.
The tradeoff lands on the provider’s side of the ledger, not the user’s. Once a model updates on one person’s prompt, it diverges from the checkpoint serving everyone else. For the provider, that means a single deployment stops behaving like one product and starts behaving like a fleet of related but distinct systems, drifting apart conversation by conversation. A GPU fleet built for test-time training needs a model instance in flight per active user rather than one shared, batched copy for an entire user base, which raises the compute and chip count needed to serve the same population.
That shift in economics selects for scale in a specific way. Pushing personalization into inference itself moves serving costs toward the capital intensity that used to belong only to training, which favors labs and hyperscalers that already run dense GPU fleets over API resellers and thin wrapper products whose margins depend on one frozen model cheaply serving unlimited concurrent users. It also draws a line inside the product map: flat monthly subscription bots do not obviously survive a compute cost that scales with usage, while agents already billed by the token or the session have room to absorb it.
The tradeoff only pays for itself where personalization compounds. A coding agent that learns a specific codebase’s conventions and recurring bugs across a long session builds memory lock-in that can justify its own per-user cost. A one-off support ticket carries no such value: a single frozen model handles it adequately at a fraction of the price, so building it a dedicated set of weights would be wasted spend. Tunguz frames the choice as which constraint a provider is optimizing against: memory limits favor the standard architecture, compute and chip limits favor test-time training, and a single stack rarely escapes both at once.
Teams evaluating agent platforms into 2026 should ask vendors directly whether a given feature runs on a shared frozen model or a per-user trained one, because that distinction will show up in the pricing tier before it shows up in any benchmark chart.
Tomasz Tunguz, general partner at Theory Ventures, published this analysis on tomtunguz.com on December 3, 2025.
