A team led by Rohan Anil has trained a 32-billion-parameter AI model across two clusters roughly 9,000 miles apart, one in the US and one in Asia, and says the result came out marginally better than training in a single room. Anil described the work in a post on the coreauto.com blog, and it targets a problem that has little to do with chips: much of the world’s spare computing power sits in pieces that nobody can use together.
The post argues that the scarce resource for big training runs is no longer the GPUs themselves. Power, grid equipment and land are what run short. It cites Microsoft CEO Satya Nadella saying “you may actually have a bunch of chips sitting in inventory that I can’t plug in,” and a Reuters report that big transformers now take over three years to be delivered. Goldman Sachs, the post says, expects just 50 to 60 percent of American data center capacity due over the next year or two to arrive on schedule, against a historical rate near 72 percent.
Meanwhile the market sells GPUs in scattered lots, often a few hundred or a few thousand at a time. Standard training software expects one big, uniform cluster that stays up, so those lots sit idle when someone wants a frontier-scale job. Anil’s pitch is that software can fix that even when concrete and copper cannot.
The test paired 160 B300 chips in the US with 160 GB300 chips in Asia. These are two different hardware generations with different setups and batch sizes, joined by a 25 Gbps link. The model was a mixture-of-experts design, where only a slice of the network activates for each piece of text. Each site trained on its own data and swapped compressed updates every 50 steps, with pieces of the model staggered so the link never saw one giant burst. A late update could still be folded in.
Given equal compute to the biggest standard run, the two-site job finished at roughly 0.2525 bits per byte, a score where lower is better, against 0.2549 for standard training. The post calls that about 1 percent better on both its evaluations. Both figures are the team’s own, and the post describes no independent replication.
A second experiment dropped the usual step where every learner averages with every other. The team divided 64 GPUs in one location into eight cells that traded model fragments in pairs, so after three rounds each cell’s progress had reached all eight. Quality matched the baseline. That test ran at one site, so it shows the method works but says little about eight distant, uneven links.
Earlier efforts paid a visible price. The post’s own history table lists an October 2026 demo that trained a 0.6-billion-parameter model over three supercomputers spread between two continents and scored perplexity of 34.7 against 28.2 for its centralized baseline, where lower is better. Anil’s team reports no such gap.
For buyers, the interesting part is who gets to train. If runs can mix chip generations and vendors and live on preemptible capacity, a lab no longer needs a gigawatt campus to attempt something large. The post says as much, then names its own catch: when a model is trained across providers, a single bad or hijacked site can quietly corrupt the model, and the damage is harder to trace back. Because the sites share weights, a problem born in one cluster can end up in every copy.
Anyone renting GPUs from several providers will soon need a way to audit the piece they did not build.
Reported by the coreauto.com blog, in a post by Rohan Anil published on 2 November 2026.
