Moonshot’s Kimi K3, due July 27, will carry 2.8 trillion total parameters yet touch only 16 of its 896 experts on any given token, engaging under 2 percent of its weight bank per forward pass. That gap between total and active capacity is not a footnote to the launch. It is the clearest evidence yet of a training strategy open-weight labs have converged on independently: buy intelligence with storage instead of compute.
Analyst Akash Bajwa traced the pattern across Moonshot’s own release history. K2, K2.5 and K2.6 all shipped on the same underlying design over nine months, a mixture-of-experts core holding roughly one trillion total parameters and 32 billion active ones. Active parameters sat in a narrow band, never straying far from the 17-to-49 billion range, for more than two years, while total parameter counts climbed roughly twentyfold since Mixtral popularized the mixture-of-experts approach, with about a third of that growth landing in the past twelve months alone.
The logic behind the choice is economic, not architectural. Holding total training compute fixed, spreading capacity across more experts drives loss lower than concentrating the same budget into fewer, larger ones, so each dollar of training compute buys more capability. Compute and memory bandwidth are rationed and expensive during training. Storage is comparatively cheap. Every additional expert lets a lab spend the cheap resource instead of the scarce ones, and it plausibly lets that lab scale capability on an axis that chip export restrictions do not directly touch.
That tradeoff does not vanish at inference. It relocates. K3’s weight file alone runs to roughly 1.4 terabytes at low precision, and Moonshot’s own guidance reportedly calls for dozens of accelerators in a single serving node just to hold the model in memory before any request is processed. Newer attention designs are cutting the memory bandwidth needed to move cached context per token, but nothing on the current roadmap shrinks the total bytes a serving node must keep resident. Sparsity solves the training-compute problem and hands the bill to capacity.
Builders evaluating these releases should sit with that distinction. For roughly two years, “open weights” and “cheap to run” were treated as nearly interchangeable claims, in part because the smaller mixture-of-experts models of 2024 and early 2025 were both compact and served on thin margins by their hosts. K3 keeps the thin margins and drops the compact size. A model with nearly three times its predecessor’s total parameter count does not get cheaper to host because it is sparse. It gets cheaper to train relative to what it can do, which is a different claim entirely.
That is why parameter count on a release announcement tells a builder almost nothing about hosting cost. A team running a handful of concurrent sessions can plausibly tier rarely used experts into system memory and approach the serving economics of a mid-size dense model. A platform batching hundreds of simultaneous requests will light up nearly the entire expert bank across that batch and needs the full model resident in high-bandwidth memory, regardless of how sparse any single token’s path through it looks.
Closed labs face the same underlying incentive to lower loss per training dollar through sparser routing, and there is no structural reason the trend stays confined to open releases. The difference is that a closed lab absorbs the resulting serving cost itself and folds it into an API price. An open-weight lab hands that cost directly to whoever downloads the checkpoint.
Teams choosing between a dense mid-size model and a sparse frontier-scale one like K3 should model serving cost at their actual concurrency level before comparing benchmark scores. The parameter count on the model card will not tell them which side of that line they fall on.
Analysis by Akash Bajwa, published July 20, 2026.
