Anthropic appears to have cut Claude’s vocabulary to roughly 16,000 tokens, down from about 50,000 in Claude 3, even as the rest of the industry keeps expanding theirs. Qwen 3.8 ships with a vocabulary near 250,000 tokens, making Claude’s the outlier by a wide margin. The gap raises a real question: what does Anthropic know that everyone else building larger vocabularies does not?
Independent researcher Sander Land reproduced a close approximation of Claude’s current tokenizer and estimated its size at around 15,000 entries. That finding, surfaced on a personal blog by developer Ian Barber, points to a specific engineering tradeoff rather than an oversight.
The likely explanation involves what a recent paper, “Lost in Backpropagation: The LM Head is a Gradient Bottleneck,” calls the softmax bottleneck. A model’s final layer has to project from its hidden dimension (D) up to the vocabulary size (V) to pick a token. For Qwen’s 2.4 trillion-parameter flagship, D is 8,192 and V is roughly 250,000. During training, the error signal from that huge vocabulary space has to compress back down into the much smaller hidden dimension, and the paper’s authors show the resulting gradient has rank capped at 2D. When V dramatically exceeds D, information gets lost in that compression, on both the theoretical and empirical evidence the paper presents.
A smaller vocabulary sidesteps the problem by keeping V closer to D in the first place. If Anthropic is deliberately trading vocabulary breadth for a cleaner gradient signal, it would explain why the company has moved in the opposite direction from Qwen, Meta, and other labs that keep growing their tokenizers.
The approach carries real side benefits beyond training signal quality. A compact vocabulary avoids the memory overhead of chunked cross-entropy kernels needed to handle a 250,000-entry projection. It also appears to eliminate glitch tokens, the bizarre outputs models sometimes produce for vocabulary entries (like the infamous “SolidGoldMagikarp”) that were included in training data but never actually trained on. With only 16,000 entries, Land’s analysis suggests, every token gets exercised often enough during training to behave predictably.
That efficiency is not free. A smaller vocabulary means more subword fragments and UTF-8 fallbacks per piece of text, particularly for rare words and non-English languages. Land’s testing found this costs Claude roughly 1.2 to 2 times more tokens per input than a larger-vocabulary tokenizer would need, which translates directly into higher attention compute and, for API customers, higher token-metered bills.
That tradeoff matters commercially as well as technically: any team benchmarking token costs across Claude, Qwen, and other frontier APIs should account for the fact that Anthropic’s smaller vocabulary inflates its own token counts on the same input text, which affects apples-to-apples cost comparisons more than raw per-token pricing alone would suggest.
Anthropic has not published its own explanation for the vocabulary size or confirmed the gradient bottleneck theory, and Land’s estimate is a reconstruction rather than a disclosed spec. Teams optimizing prompt length or comparing per-token costs across model providers should treat Claude’s higher token counts on identical inputs as a structural feature of this design choice, not noise, when modeling their next quarter’s inference spend.
Reported by Ian Barber on 24 August 2026, based on independent tokenizer analysis by researcher Sander Land.
