Lon Lundgren spent six weeks reverse engineering his own Claude Code usage data after he felt the tool get noticeably worse to use. What he found, in a long essay he posted on X titled “The Inference Gap,” is a gap between the model you are told you can access and the amount of actual reasoning that model performs on your behalf.
Lundgren’s starting point was Anthropic’s rollout of Fable 5, the company’s first “Mythos” class model. The US government briefly blocked access over safety concerns before Anthropic restored it on July 1. Anthropic then pushed back its own deadline for making the model permanently available to subscribers three times, from July 7 to July 12 to July 19, before settling on July 20. Lundgren says his experience of the model’s performance dropped sharply in the days that followed: instructions were ignored, reasoning grew inconsistent, and results he had gotten easily in the model’s first weeks became difficult to reproduce.
Rather than assume he had been quietly downgraded to a cheaper model, a common complaint after nearly every frontier launch, Lundgren built a proxy to capture live data from the wire and spent from August 1 through early September parsing transcripts and output tokenization. The resulting corpus ran to 43,261 model invocations, with 7,853 full assistant turns captured across three machines, two subscription accounts and 213 recorded sessions over 65 active days, pulled straight from live wire traffic rather than any dataset Anthropic had published.
His central finding is about “thinking tokens,” the reasoning the model performs before answering. Even at the two highest user-selectable effort settings, xhigh and max, the median invocation in his corpus produced just 123 thinking tokens, and 39.2 percent of invocations produced none at all. He compares this to reasoning-token estimates he reconstructed for Anthropic’s own published ARC-AGI-2 and HLE benchmark results at the same effort settings, which cluster in the tens of thousands of tokens. By his math, reaching that range from his observed median would require roughly a 230-fold increase in reasoning output.
Lundgren also found the amount of reasoning delivered was not constant. Comparing July to August across five different weighting methods, he measured declines in median thinking tokens ranging from about 22 percent to 51 percent, largest at the individual invocation level. He is careful to note his data cannot identify the cause: capacity management, routing changes, and adaptive effort logic are all consistent with what he observed, and he is not accusing Anthropic of a specific practice.
This is one user’s self-published, independently reconstructed analysis, not a peer-reviewed study or a disclosure from Anthropic, and it relies on inference from wire traffic rather than data the company has confirmed. Anthropic has publicly described effort settings like xhigh and max as a “behavioral signal” rather than a fixed token budget, language that leaves room for exactly the variability Lundgren logged.
Lundgren’s proposed fix is disclosure: he wants providers to report realized reasoning tokens per invocation and per turn, the share of calls that receive zero thinking, and how those distributions shift over time, arguing that a model name and an effort setting no longer describe what a user actually receives.
For any team benchmarking a model against a vendor’s published scores, Lundgren’s numbers are a reason to log your own reasoning-token traces in production rather than trusting a single onboarding test run.
Based on Lon Lundgren’s essay “The Inference Gap,” posted on X on September 18, 2026.
