Hiring a team of AI agents instead of one cost between 1.8 and 5.1 times as much on Vals AI’s coding test, and the extra spend bought a score gain that could be told apart from chance in only one of four comparisons. Vals AI published the results on October 9. Vendors are selling delegation to subagents hard right now, so a paid test that mostly comes back empty is worth a careful read.
Vals AI sells model evaluation, and the test ran on its own benchmark. Vibe Code Bench asks an agent to build a complete web application from a product spec, then sends browser agents through the finished app to click its workflows. Vals ran GPT 6 Sol and Claude Opus 5.5 on 50 apps, with every model run alone and again with subagents, at medium and at max reasoning effort. In a team, a lead agent could hand work to as many as five subagents at a time and then integrate what came back.
The one clear win belongs to Sol at medium effort. Its team averaged 84.9 percent against 77.6 percent for the single agent, a gain of 7.3 points (p = 0.005, with a 95 percent interval of +2.3 to +12.4). Vals traced it to a specific habit: on the nine apps where the team came out 20 points or more ahead, the lone agents reported passing tests while whole features were never built. The team lead checked the separately built pieces against each other and caught mismatches, such as a form whose fields disagreed with the database. The gain concentrated in the harder apps, where it averaged about 15 points; the easier apps saw roughly 1.
The other three comparisons moved by between minus 0.3 and plus 3.4 points, with p-values from 0.16 to 0.81. “Not statistically significant” means the test could not separate those differences from luck at this sample size. It does not mean the teams performed the same. Opus at max effort gained 3.4 points, but the interval runs from minus 1.4 to plus 8.3, which leaves room for no effect and for a sizable one. Fifty apps is a modest sample, and every one of the eight setups was run a single time, so the intervals capture variation between apps, not between repeated runs. A null result this wide is weaker than it sounds.
The price spread is the most usable number in the study. Teams cost 2.5 times as much at medium effort and 1.8 times at max effort for Sol, and 2.2 times and 5.1 times for Opus. Vals attributes most of the extra spend to cached input: every subagent keeps its own copy of the spec and its working state, and that copy is re-sent on each model call. At max effort, the median Opus app saw its subagents consume 224 million cached tokens, where the single agent used 55 million. The Opus max team averaged $122 per app, compared with $23.77 for its single agent, and four of its runs dragged past the four-hour mark. That $122 includes one run that ran out of time, whose cost of roughly $189 was estimated from token counts.
Sol’s costs use list prices and leave out long-context surcharges, so cost comparisons across the two models are soft. Vals draws a narrower conclusion than the headline number suggests: whether a team pays off depended on the model, the task and the budget. Higher reasoning effort looked like the better lever for Sol. Max effort lifted its single agent by 11.4 points over medium (p below 0.001), more than the team did, and landed at 89.0 percent for $4.82 per app against $3.07 for the medium team. For Opus, neither lever mattered much: the medium single agent scored 91.5 percent for $4.08, while the best score in the study, 93.2 percent from the max-effort team, cost thirty times as much and did not clear the significance bar against it.
The scope is small. Two models, one benchmark, and one kind of work: building web apps. Nothing here says how agent teams behave on research, analysis or any non-coding task. The two models also ran through different harnesses, so cross-model comparisons mix the model with its tooling, and the team arm tested subagents plus a delegation prompt rather than subagents alone. Vals itself points to work with many independent parts, such as large codebases or broad research questions, as the place to look next.
This result does not argue that teams of agents are a bad idea, and it should not be read that way. It argues for a different first question when someone quotes a team price: what does one agent at higher reasoning effort score on the same job, and what does that run cost?
Vals AI, “Do Agent Teams Pay Off? A Case Study on Vibe Code Bench,” by Katrina Drozdov, published October 9, 2026.
