Google engineers have built a way to make the two halves of a reinforcement learning loop compute exactly the same log probabilities, down to the last bit, on TPUs. The work, called Zero-TIM (zero train-inference mismatch), was published November 16 on the Google Developers Blog by Yuxuan Zhang, Lin Chai, Lance Wang, Tianshu Bao, and Weiren Yu. It was verified across 64 TPU v5p chips arranged as a DP8 by TP8 mesh, and it ships as part of Tunix, Google’s open-source JAX library for LLM post-training.
The problem is quiet but costly. In RL for language models, a sampler generates tokens and records how likely it found each one (the log probability, or logprob). A separate trainer later re-scores the same tokens in big batches to compute gradients. Even with identical weights, the two code paths can disagree on those numbers. The training schedule looks on-policy, meaning the model learns only from its own current behavior, yet a hidden gap opens up and grows across multi-turn trajectories.
The root cause is arithmetic order. Summing floating-point numbers is not associative, so adding the same values in a different grouping can round differently. Google’s example uses bfloat16, a compact number format with little precision: near 2.0, an input of 0.0078125 is half the distance to the next representable value, so adding such inputs one at a time discards each, while grouping them first reaches the next value. Matrix products, normalization, and tensor-parallel shards all contain sums like this, and the sampler and trainer ordered them differently. The weights matched. The arithmetic did not.
The team defines a strict contract: the sampler’s decoding, the serving engine’s prefill, and the trainer’s forward pass must all return identical raw bytes for every sampled action, with no tolerance. Adding prefill as a middle checkpoint tells you where a mismatch enters, which a direct sampler-versus-trainer comparison cannot.
Getting there took four steps. Custom Pallas kernels (JAX’s low-level kernel language) fix the accumulation order inside each shard and accumulate in float32. Cross-shard sums happen in an explicit, fixed rank order rather than whatever the collective operation picks. The final logprob scoring is standardized across all three paths. Serving kernels are wrapped in custom differentiation rules so the trainer reuses the exact serving forward pass, and the XLA compiler flag —xla_allow_excess_precision=false keeps surrounding operations from sneaking in extra precision.
The reported test is modest. Google ran 200 steps on FrozenLake, a synthetic grid puzzle, with grids from 2 by 2 through 9 by 9 and up to five interaction turns, starting from a public Qwen3-8B model, a GSPO-token loss, and leave-one-out advantages. Zero-TIM logged zero mean absolute logprob difference at every step, and its solve rate rose. A standard run and one with truncated importance sampling at the token level (TIS), which reweights samples to compensate for mismatch, kept nonzero differences. Google says Zero-TIM reached higher observed solve rates. Those are single-seed runs on Google’s own benchmark, and the post does not report results on a production-scale model or task.
The post also places the work in a crowd. It cites vLLM with TorchTitan and RadixArk with LMSys Miles as earlier efforts that used batch-invariant kernels and bitwise alignment to prevent drift during policy optimization. What Google adds is a JAX, XLA, and Pallas version for TPU, and it says support for linear-attention and Mixture-of-Experts architectures is being extended.
The practical cost is not stated. Fixed-order kernels and custom backward rules usually trade some speed for determinism, and the post gives no throughput or memory figures. A team running RL on TPUs should test Tunix’s Zero-TIM prototype on its own workload and measure that overhead against the stability gain before replacing importance-sampling corrections.
Google Developers Blog, “Unlocking Stable On-Policy RL on TPU with Bitwise-Zero Train-Inference Mismatch,” published November 16, 2026.