Microsoft Research has published a preprint describing an architecture change that keeps a language model’s internal reasoning alive between generated words instead of discarding it after every token. The paper, titled “Full-bandwidth transformer,” targets a structural bottleneck present in every transformer deployed today. It has not been peer reviewed, and the results it reports come from a research-scale training run, not a shipped product.
Here is the bottleneck the authors describe. When a transformer generates a token, it runs a full forward pass through every layer, building a more detailed internal representation as information moves up the stack. At the very top, that representation gets collapsed into a single output: the next token, sampled from a probability distribution. Only that token, a compressed integer, feeds back into the base of the network to begin the next step. The rich vector that produced it, known as the top-layer hidden state, gets discarded.
The authors call their fix latent feedback. Instead of throwing away the top-layer hidden state, the model combines it with the embedding of the token it just sampled, using a small trained component called a gated linear unit, and passes that combined vector into the next step alongside the normal input. The model then carries forward not just what it said but a compressed trace of how it got there, letting computation that never turned into words keep accumulating across steps with a fresh pass through the full depth of the network.
Building this cleanly is harder than it sounds. Standard transformer training depends on teacher forcing: every position in a training sequence can be computed at once because the model never needs its own prior output, only the correct answer supplied by the training data. Latent feedback breaks that assumption by chaining each step’s input to the previous step’s internal state. The authors address this with what they call a scheduled multi-pass objective, introducing latent feedback only late in pretraining and mixing in a small share of deeper, sequential passes to keep training stable.
The team trained 1 billion parameter models with latent feedback out to 400 billion tokens and measured them against standard transformer baselines. Per the preprint, latent feedback lowered validation loss and lifted scores on five-shot language model evaluation, on math and coding generation, and on instruction-tuned performance. The authors say the gain arrives with almost no added cost per decoded token, and that the resulting models perform close to standard transformers trained on about 1.5 times as many tokens. They also report the models needed fewer tokens of reasoning to hit equal or better accuracy, a sign the recycled hidden state does some of the work that would otherwise require writing out more text.
That 1.5x token-efficiency figure is the authors’ own comparison against baselines they built, and the test stops at 1 billion parameters. Architectural changes that show clean gains at that scale have a mixed record once labs push them to the tens or hundreds of billions of parameters where frontier models actually compete; this preprint offers no evidence either way at that range. The listing does not name a lab timeline for testing the idea beyond the reported 400 billion token run, and nothing in it indicates the technique has shipped in, or is planned for, any production model.
Engineers evaluating pretraining architecture choices should treat this as a signal worth tracking through peer review and larger-scale replication, not as a technique ready to adopt today. Anyone modeling inference cost against KV cache size should note the authors’ claim that the standard KV cache and decoding overhead stay essentially unchanged, since that detail is what would decide whether latent feedback survives contact with production-scale serving.
This article is based on the arXiv preprint titled Full-bandwidth transformer by Xi Wang and seven coauthors at Microsoft Research, posted to arXiv as 2608.08888 on August 9, 2026.
