Independent researcher Yifan Zhang published a technical report proposing an architecture he calls the Recurrent Looped Transformer, or RLT, built around a single encoder-decoder pair that keeps one running state alive from the first prompt token through the last response token. Zhang posted the work on a personal GitHub Pages project site rather than through a lab or a peer-reviewed venue, and he is upfront that the design is a proposal, not a validated result. His own report states that “realized reasoning gains, hardware efficiency, and RL scaling remain to be established.” That line is the story, not a footnote to skip past.
The idea is to stop treating pretraining, fine-tuning, sampling and reinforcement learning as separate modes stitched onto a transformer after the fact. RLT routes all four through one shared state transition instead. An encoder scans already-known tokens in parallel and builds a global memory of keys and values from them. A decoder then carries forward its last hidden output plus a rolling attention cache, layer by layer, so the computation keeps extending rather than restarting each time a prompt hands off to a response.
Depth, in Zhang’s framing, is not a fixed count of layers run once per token. It grows with how long the sequence gets: the recurrent path a token’s computation travels lengthens in step with the number of tokens already processed, even though the work done at each individual step stays constant. Zhang specifies a version with 48 layers on each side of the encoder-decoder split, which puts roughly 96 logical computation steps behind every generated token, a figure that climbs further as the exchange continues. He draws a careful line around what that means: “Infinite depth refers to an extensible temporal path, not infinite work within a token.”
The reinforcement learning piece is the most speculative of the three arguments in the report. It calls for reconstructing the entire prior sequence, cached attention states included, using the model’s present parameters before that history gets replayed for a policy update. The goal is to make the probabilities recorded for past actions match the sampler that actually produced them at the time. That targets a genuine bookkeeping headache in off-policy reinforcement learning for language models. Whether the approach holds up at production training scale is untested; the report does not claim otherwise.
AI Insiders covered Sebastian Raschka’s analysis of looped transformer designs last week alongside the GPT-6 Astra release, and RLT sits in that same open question: whether looping computation inside a transformer, rather than piling on more layers, is a real path to deeper reasoning. Raschka treated looping as a pattern worth watching rather than a settled technique. Zhang’s report pushes that bet further, aiming to unify how a model trains and how it serves requests under one mechanism, instead of only adding extra reasoning passes at inference time.
The report carries no benchmark scores, no training curves and no independent replication. Anyone trying to judge whether a recurrent, stateful transformer beats a standard one on coding or reasoning tasks has nothing here to measure against. RLT is worth tracking as an architecture proposal. It is not yet a result anyone should cite as evidence that this design works, and Zhang’s own text says the evaluation that would prove it still needs to happen.
Based on Yifan Zhang’s technical report “Recurrent Looped Transformer,” published on his project page in September 2026.
