Microsoft says its giant AI model beats rivals, on scores only it has run

Microsoft has finished training MAI-Base-2, a very large AI model, and says it beats every openly released competitor it tested while costing a fraction of the computing power. Microsoft AI, the company’s in-house model lab, made the claim on its own news blog on October 15. Read it as a progress report from one side of the table.

The model is a “mixture of experts” design: 2.8 trillion parameters in total, of which 80 billion are active on any given piece of text. It is a base model, meaning it has finished learning from raw data but has not yet been taught to behave like an assistant. Microsoft says that post-training is under way and promises more results “soon”. It gives no release date and no price.

What the test measures. Microsoft scored models on bits-per-byte, a measure of how well each one predicts held-back text it never saw in training. Lower is better, and the measure works across models with different tokenizers. The post says MAI-Base-2 scored lower than every model it tested. The words “it tested” carry the weight. The rivals are DeepSeek-V4-Pro-Base, Kimi K3, Qwen3.8-2.4T-A95B, and MiMo-V2.6-Pro-RL, and no closed model from OpenAI, Google, or Anthropic appears in the comparison.

Only one of those four is a true base model. For the other three, Microsoft says only post-trained versions exist, so it ran those and notes that post-training “may affect these evaluations”. That is a handicap of unknown size on three of the four rivals.

Microsoft did not simply grade its own homework. The post says the STEM, math, and question-answering tests were commissioned from outside vendors to reduce the chance of the model having seen the answers, and every test set was screened against the training data. The code test is different: it uses Microsoft’s own codebase. The vendors wrote the questions, but Microsoft ran the models and published the scores, and the post cites no outside reproduction.

The efficiency claim. Microsoft says its training recipe is 4.2 to 12.3 times more efficient than DeepSeek-V4-Pro-Base, measured in raw arithmetic operations. Microsoft computed that figure itself. It trained a family of smaller models, fitted curves to their results, and estimated how much compute its recipe would need to match DeepSeek’s score. That is an extrapolated estimate, not a head-to-head training race, and it leans on the compute figure for DeepSeek’s run. The post also notes that DeepSeek’s model was trained on far fewer operations than MAI-Base-2.

The hardware figures are more concrete. The run used 16,000 GB200 chips for 38 days across August and September 2026. Average utilization was 21% of the chips’ theoretical peak, and training made progress 89% of the time. Microsoft attributes the saving to its data and modeling choices, not to unusually high hardware utilization.

What an engineer could take from it. Two details travel. First, Microsoft fixed memory use in the expert layers so overloaded experts no longer crash a run or drop tokens, lifting those layers from 24% to 35% utilization. Second, it clipped gradients on individual examples rather than whole batches, and the same mechanism flagged a bad training sample: a code file whose license header was cut off mid-sentence, which exposed a preprocessing bug.

The data and the business angle. Microsoft says the model was trained on “organic data” with no distillation, meaning no learning from other companies’ models, and that its own pipelines handled all of the training tokens. The corpus grew by roughly 36% over its predecessor, with images making up 18% of a 50 trillion-token mixture. For web content it says its crawler respects robots.txt and excludes certain sources. The post says nothing about licensing deals with publishers or rights holders, and it does not list what the web data contained.

That provenance is the sales pitch. Microsoft says it will license the model weights to “trusted partners” through Foundry, its enterprise AI platform, and describes owning the model pipeline as a distinctive offering. The post says only that. It does not mention any other lab or any contract, so what this means for Microsoft’s dependence on others is inference, not something Microsoft states.

For anyone buying AI through Microsoft, the question for the next quarter is the post-trained version: until it ships with independent benchmark numbers, MAI-Base-2 is a claim about a recipe, not something to plan around.

Based on Microsoft AI Lab Notes blog post of 15 October 2026