Microsoft's decision model is priced to grade other AI's work

Microsoft is selling a model at $0.042 per million input tokens whose most interesting job is marking other AI systems’ homework. Microsoft-Decision-1, a small model that returns a probability for every answer on a fixed menu instead of writing prose, went live October 10 in Microsoft Foundry (Microsoft’s catalog and hosting service for models) and on OpenRouter, the marketplace that puts many vendors’ models behind one API key. Output tokens are free, according to Microsoft’s launch post.

Other vendors shipped similar models this week, so the category needs no introduction. What sets Microsoft’s apart is the list of answer formats. Yes/no, multiple choice, and ratings are the expected ones. Microsoft also lists grading against a written rubric, applied to both chatbot answers and the actions an agent takes. That turns the model into the judge in an LLM-as-judge pipeline.

That is where the money is for anyone building evaluations. Grading every output of every test run with a frontier model is slow and expensive, and at $0.042 per million input tokens Decision-1 is cheap enough to grade all of it. On the post’s own median latency figures, Decision-1 answers in 85 milliseconds against 3,010 for GPT-6 Sol.

The post’s grading evidence is thin, though. The Copilot team found the model “competitive with GPT5.6 Luna” at measuring chat and agent responses, and 100 times faster, with no accuracy figure attached. Microsoft Discovery, the company’s science-agent product, scored it 46 times more consistent than an LLM-based grade when an agent revises its plan from rubric marks. Consistency is not correctness. A grader that gives the same wrong mark every time is perfectly consistent.

So who checks the grader? The post reports no agreement rate with human graders on rubric tasks. It does report calibration, meaning how closely stated confidence tracks how often the model is right. Decision-1 scores 92.2 out of 100 there, third among the six models scored, behind Jev 1.13.0 at 93.7 and Quyet-1.0-Large at 93.1. A judge that is confidently wrong on a fraction of cases will quietly pass bad agent actions, and nothing in the launch material measures that.

Microsoft’s headline says the model “delivers top performance in latency and quality” and outperforms LLMs and other decision models. Unusually, the post names its field. On average accuracy across 36 benchmarks (147,137 questions, some of them private), Decision-1 scores 83.5%. Jev 1.13.0 follows at 82.3%, Quyet-1.0-Large at 81.9%, OpenAI’s GPT-6 Luna Decisions at 79.4%, and H2O-Lightning-4B v1.1 at 77.2%. The lead is 1.2 points. Microsoft ran the comparison, and added Jev’s numbers in an update after publication. GPT-6 Sol, the large model used for most speed claims, has no accuracy score in the chart, so the claim to beat LLMs on quality is not shown for it. Microsoft measured its own latency through Foundry, while rivals’ figures come from the JevBench leaderboard.

The practical facts are partial. Microsoft built the model by post-training Qwen3.5-9B, an open-weight base, and says it will later rebase it on Microsoft AI and OpenAI models. It does not say whether the weights can be downloaded, so treat this as API-only. It states no context window. Foundry implies Azure billing and enterprise controls; OpenRouter implies pay-as-you-go access with no Azure account, and puts the model one click from its competitors. The post does not say whether the price is the same on both.

Microsoft already resells OpenAI’s models at frontier prices. Building its own small model for the cheapest, highest-volume call in an agent stack suggests Microsoft sees little margin in reselling that call and some in owning it. Microsoft’s own arithmetic shows why: a 100-millisecond delay repeated across 20 chained decisions costs two seconds. Three vendors converged on the same narrow task at once because agent loops make hundreds of such calls per job, and none of them wants to be paid frontier rates for a yes or a no.

If you run LLM-as-judge today, score a few hundred outputs with humans and with Decision-1, and compare. Microsoft’s Xbox Research team measured the model at 200 times cheaper than GPT-6 Sol on survey labeling, but labeling is not grading, and a cheap judge only helps if its marks match yours.

Microsoft, “Introducing Microsoft-Decision-1, our model for fast decision-making,” Microsoft Command Line blog, published October 10, 2026.