General Reasoning, an AI lab that publishes its research at gr.inc, has laid out how it trains “swarms”: groups of AI agents that cooperate on one job. Its answer is to reward a team for how much it gets done before a deadline, then let the agents talk to each other freely. The lab calls the recipe reinforcement learning from collective rewards, or RLCR, and its first model built this way is Avalon-M1.
Start with why this is hard. When a lab trains a single reasoning model, the reward is simple: get the answer right. Over time the model discovers that checking its work and trying alternatives pays off, so it thinks longer. Nothing in a plain task reward, though, tells a group of agents to work side by side. A team that finishes in six hours scores the same as one that finishes in one. The General Reasoning team argues that the training setup has to make faster work pay, because parallel effort is the only way to get there.
The lab compares two ways to do that. One is an explicit penalty for elapsed time. DeepSeek reportedly used a version of this for the agent teams in its V4.1-Flash model, though it did not share the formula. General Reasoning chose the second route, a hard time limit. When a run hits the cap, the score is whatever progress exists at that moment, so the team has to get as far as it can before the clock stops. That only works in tasks where partial progress can be measured at any time, such as the number of software tests passing or the returns on a trading simulation, and the lab built its training environments around that property.
The deadline is not fixed. It starts at about two hours and stretches to just over five by step 200 of training. The reasoning is that a team pays a coordination cost up front, and a lone agent will beat it early on. Longer horizons give the team room to earn that cost back.
Communication is the other half. Avalon-M1 is led by a CEO agent with the power to hire and fire up to eight specialists, and any agent can message any other, whether by channel, private note, or a shared archive. The lab says it wants these tools kept simple so agents can find their own ways of cooperating. It also points out that existing systems build in structure. Kimi’s Agent Swarm trains only the orchestrator and keeps its sub-agents frozen, while OpenAI’s Navier-Stokes effort split its agents into groups, each briefed on its own variant of the task.
The results are the lab’s own, and the full paper is still weeks away. In its report, teams of five to eight hires began below the average run for useful work and ended near double it by step 200. Agents also spent more effort on communication as training went on, though the lab concedes that volume alone does not prove the talk was useful. At the same total output budget, a single agent took four times as long as the team. On the long-horizon EdgeBench test, an eight-hire team passed the single-agent baseline at about 2.4 hours.
The lab is candid about what it has not solved. Avalon-M1 trained with at most nine agents, while swarms in the wild now run to hundreds or thousands, and nobody knows which habits survive that jump. Credit assignment is also crude: one shared score is spread over runs that last hours and can burn through millions of tokens.
For anyone buying or building agent teams, the number to ask for is time to a given score at a fixed budget, because a team that costs the same as one agent but finishes faster is the only version worth the extra moving parts.
Reported by General Reasoning on its blog at gr.inc (undated post “How to Train a Swarm”).
