OpenAI’s chief scientist Jakub Pachocki publicly pushed back this week on a scoop from The Information claiming GPT-6 Astra relies on “looped transformers,” an architecture that reuses the same weights across multiple passes. Pachocki wrote that the computation graph depth of the company’s current frontier models, Astra included, is “within a factor of two of GPT-4.”
Sebastian Raschka, the machine learning researcher and author who writes the Ahead of AI magazine, used that clarification as the anchor for a longer technical breakdown of what looped transformers actually do and whether they would explain a separate, more sensitive claim: that Astra’s reasoning traces have become harder to monitor.
Raschka is explicit that OpenAI has not confirmed the architecture. He calls it “highly likely” based on three things: The Information’s reporting, prior research showing the technique works, and Pachocki’s own statement, which he notes “doesn’t confirm the looped transformer architecture explicitly” and could just as easily describe a model with twice as many conventional blocks.
The mechanism itself is not new. A looped transformer passes hidden states back through the same stack of blocks multiple times instead of adding new blocks with new weights, an idea Raschka traces to the 2018 Universal Transformer paper and shows recurring in ByteDance’s Ouro, the open-weight model Nanbeige4.2-3B, and the Mixture-of-Recursions approach. The appeal is straightforward: looped transformers can lift modeling quality without raising the compute budget, provided the model is large enough, and without adding storage for new parameters, though they still require the same forward and backward compute as separate blocks would.
Where Raschka pushes back hardest is on the interpretability question. The worry, as he frames it, is that a model doing more computation internally through looping might need fewer visible “thinking tokens,” making its written chain of thought a less faithful record of what it actually did. He does confirm that Astra uses fewer output tokens than its predecessor at matched accuracy on some benchmarks. But he argues that pattern also shows up between older models of different sizes that nobody treats as an interpretability problem, and that a shorter trace more plausibly reflects a model that makes fewer mistakes and backtracks less.
He also points to language in Astra’s own system card acknowledging reduced monitorability of its reasoning traces relative to its predecessor, tied to shorter and less informative outputs. Raschka reads that as consistent with general trace-length effects rather than evidence that looping specifically is the cause.
The distinction matters beyond one model. If shorter reasoning traces are simply a byproduct of stronger models solving problems faster, monitoring tools built around trace length as a proxy for depth of thought need to be rebuilt around a different signal. If an architecture choice is instead quietly degrading the readability of chain of thought across the industry, that is a safety-research problem independent of any single lab’s benchmark scores. Teams building interpretability or alignment tooling on top of frontier reasoning models should treat trace shortening as a signal to investigate directly, rather than assume it tracks a specific architectural cause without independent confirmation.
Sebastian Raschka wrote this analysis in his Ahead of AI magazine on September 9, 2026.
