Replit’s research team says a swarm of 32 Claude Opus 5.5 agents sped up a set of NumPy workloads 18.9 times in two hours, while a single Opus agent managed 3.7 times. The result comes from Optimization-Bench, a performance-engineering suite Replit published on November 19. The more interesting finding sits next to it: the strongest solo model in the test, GPT-6.2 Astra, got worse as its team grew.
The setup is simple to describe. Replit gave each agent 8 vCPUs, 16 GiB of memory, and two hours to make real open-source libraries faster without changing their behavior. Teams of 1, 8, 16, and 32 agents ran on six models, and the coordinator received no instructions on how to organize the work. Replit reports full results for four of its 14 libraries: NumPy, SQLite, SQLGlot, and Git.
Replit chose this task deliberately. Build benchmarks such as ProgramBench cap the score at a full pass, so a team can only finish sooner. Speedups have no ceiling, and agents that skip sharing results end up repeating each other’s dead ends. That makes coordination show up directly in the number.
The Anthropic and OpenAI models organized themselves differently. According to Replit, Opus 5.5 sent 82 percent of its messages to a shared board, mostly in one thread per workload where workers claimed pieces and posted findings. Opus workers averaged 1.4 direct messages to the coordinator. Workers on GPT-6 Astra and GPT-6.1 Sol averaged about 22 and 20.
Direct messages have a structural cost: they funnel into a single inbox. On SQLGlot, the GPT-6.1 Sol coordinator went from 356 to 631 messages received when the team doubled from 16 to 32 agents. GPT-6 Astra’s coordinator went from 346 to 615 on NumPy. Scores barely moved, which suggests the coordinator had become the bottleneck, not the workers. GPT-6.2 Astra ran into the same ceiling, and its best swarms were mostly its 8-agent teams.
Solo rank turned out to be a poor guide. GPT-6.2 Astra was the best solo agent on all four libraries, and on NumPy it beat a lone Opus agent by a factor above three. Yet Opus swarms outscored it on NumPy, SQLite, and SQLGlot. Replit says Opus and Claude Sonnet 5.5 swarms beat their own solo runs every time, by a median of 1.30 times at 16 and 32 agents.
The shape of the codebase mattered as much as the model. NumPy offers 13 mostly separate workloads and produced the biggest jump. Git’s seven workloads all live in the same 15 files, so extra agents had little to divide and Anthropic’s swarms struggled to beat their solo runs. On Git, GPT-6.2 Astra at least tied Opus for every team size tested.
One concrete win shows what the gains look like. On NumPy’s searchsorted function, the Opus swarm made the search process 256 keys at a time so the data stayed in CPU cache, which Replit says made float64 inputs with many keys 8.7 to 13.1 times faster with identical output.
Two cautions apply. Replit sells swarm products (its Missions feature launches a team of agents against a goal), so the company has a stake in the conclusion that teams can pay off. And the models ran at medium reasoning effort inside Replit’s own harness, so a different orchestration setup could reorder the results. The benchmark and its code are open, which lets anyone check.
The practical takeaway is that orchestration is now a model-selection question separate from coding ability. A team choosing which model should coordinate a fleet of agents should test that role directly, with swarms of 16 and 32, and watch the coordinator’s inbox count before trusting a solo leaderboard.
Replit, “Throwing Headcount at the Problem” by Michele Catasta, published November 19, 2026 at frontier.replit.com.