Claude Opus 5.5 retained 80% of its benchmark score while costing just 16% as much once Vals AI dropped it to the lowest reasoning setting. Vals AI, which runs independent benchmarks of AI models, published the measurement on its blog on 3 November 2026, in a post by Derrick Liang covering seven models from seven labs.
Reasoning effort is a dial that tells a model how long to think before it answers. Higher settings let it work through more steps behind the scenes, and every step is billed. Vals normally runs its leaderboard at each model’s top setting, which is “max” or “xhigh” for six of the seven models. Gemini 4 Argon has no effort level in Vals’ leaderboard runs, so its high setting stood in.
The test used the Vals Index, which bundles eight benchmarks covering money, software, law and tax tasks. Each model ran at its lowest, a middle, and its leaderboard setting. The table below shows the lowest setting as a share of the leaderboard setting, score first and cost second:
- Gemini 4 Argon: 99% of the score, 84% of the cost
- DeepSeek V4.1 Flash: 89% of the score, 50% of the cost
- GPT-6 Astra: 88% of the score, 18% of the cost
- Grok 4.7: 86% of the score, 16% of the cost
- GLM-5.3: 81% of the score, 53% of the cost
- Claude Opus 5.5: 80% of the score, 16% of the cost
- Muse Spark 1.3: 77% of the score, 29% of the cost
Four models held 80% or more of their score while paying half the price or under. Gemini 4 Argon is the odd one out: its lowest setting saved little, but its middle setting scored slightly above its leaderboard run at 78% of the cost.
Zoom in from the whole index to individual benchmarks and the picture gets sharper. Across 56 model-benchmark combinations, a cheaper setting matched the leaderboard score in 22. Nothing cheaper scored significantly higher, and 33 cheaper runs scored significantly lower. On the full index, none of the seven reached its leaderboard score at the lowest setting, so the savings are real but not free.
Tasks differ a lot in how much they reward extra thinking. Going from the lowest to the highest setting lifted Legal Research by a median of 19.2 points, Code Migration by 15.6 and Terminal-Bench 4 by 12.1. Finance Agent 2.1, HLAB and EMB each moved under 6 points. A team running mostly legal research has a different cost curve than one running mostly finance questions.
That gap suggests choosing a setting per task instead of one setting for everything. Vals tested the ceiling on that idea by picking, with hindsight, the best setting for every task. The median gain over the leaderboard setting was 6.0 points, while costing a median 64% of the leaderboard run. Running the lowest setting everywhere costs 29% of the leaderboard price, so even perfect routing is not the cheap option. Because an actual system would have to pick blind, Vals calls 6.0 points an upper bound.
The caveats are Vals’ own. Every score comes from about half the tasks in each benchmark, most benchmarks had one run per setting, and a “no significant difference” result can hide a small real gap. Most leaderboard runs date from earlier months while the other settings ran in October, so provider-side changes could skew the comparison.
For anyone paying per token on a high-volume workload, the cheapest experiment this quarter is rerunning your own evaluation at the lowest setting. If your task looks like Finance Agent or EMB, expect to give up very little; if it looks like Legal Research, expect to pay for the thinking.
Reported by Vals AI on its blog, in a post by Derrick Liang dated 3 November 2026.
