RadixArk shipped Miles v0.1 on 18 August, a production-hardened update to the reinforcement learning framework it open-sourced on 1 July and that AI Insiders covered at the time. The new version targets the single biggest source of wasted GPU time in agent training: the pause between finishing a training step and getting the updated model back to the workers generating the next round of rollouts.
That pause is expensive because of how synchronous RL training works by default. The trainer waits for every trajectory in a batch to finish before it can compute a loss and update the policy, and the rollout fleet then sits idle while the optimizer works. In agentic tasks, where a session can run through dozens of tool calls and vary wildly in length, a handful of slow trajectories can stall an entire batch. According to a post on LMSYS’s blog, that dynamic is what makes long-context and tool-use reinforcement learning inefficient at scale.
Miles v0.1’s fix is a fully asynchronous loop. SGLang inference engines keep generating trajectories continuously, the trainer consumes whatever completed groups are ready, and weight updates flow back to the fleet without halting generation. Scheduling runs per trajectory instead of per batch, so a finished sample frees its slot immediately rather than waiting on slower siblings, and a buffer between rollout and training lets the two run at different speeds without either one blocking the other.
Reinforcement learning post-training for agents, work that requires tool calls, sandboxed execution, and long multi-turn context, has mostly stayed the province of labs with the systems engineering staff to build asynchronous pipelines, weight-sync infrastructure, and fault-tolerant evaluation from scratch. Miles packages that machinery: isolated sandboxes per rollout episode with integrations for environment providers such as E2B, Daytona, and Modal, a routing-replay mechanism that keeps mixture-of-experts training numerically consistent with what the rollout engine actually computed, and support for both NVIDIA and AMD hardware.
Whether that closes the gap or just makes the easy part visible is the open question. Some of the weight-synchronization work is genuinely hard-won: peer-to-peer transfer cut update time for a 1-trillion-parameter Kimi-K2 checkpoint from 53.3 seconds down to 7.2 seconds, and disk-delta updates can shrink a weight payload from 62.4 GB to under a gigabyte. But the framework’s most exact numerical-alignment feature, Zero-KL Alignment, which is meant to erase the gap between what the rollout engine produces and what the trainer sees, is scoped today to the Qwen 3 family alone. That is a narrower footprint than the day-zero support Miles claims for models such as DeepSeek-V4 and Kimi-K3.
The reference run in the post trains a 744-billion-parameter GLM-5.2 model on terminal-use coding tasks across 64 NVIDIA GB300 GPUs, split evenly between rollout and training. Each training step took roughly four and a half minutes, with rollout weights trailing the trainer by 1.7 steps on average. That is not infrastructure a solo researcher spins up on a weekend. It unblocks teams that already have multi-GPU budgets but lack a dedicated systems group to build async scheduling and sandboxed environments in-house, not teams priced out of frontier compute entirely.
Organizations already running agent RL on internally built infrastructure should benchmark their straggler losses against Miles’s async throughput before their next training cycle. Teams without that infrastructure now have a credible starting point, provided they can supply the GPU fleet the reference run assumes.
Reported by LMSYS on its blog, 18 August 2026.
