Nathan Barry has built an inference pipeline that produces bit-identical output across GPUs, CPUs, and batch sizes, by dropping floating-point numbers from the model entirely. He describes the project, DetLLM, in a November 12 post on his blog nathan.rs. The work was accepted as a paper at two NeurIPS 2026 workshops.
The problem he targets is familiar to anyone who has tried to reproduce a model run. Thinking Machines showed that sending one prompt 1,000 times at temperature 0 produced 80 different answers. The cause is that float addition is not associative: summing a, b, and c in a different order can round differently. A GPU adds thousands of values per token, and the order depends partly on how many other requests share the batch.
Thinking Machines fixed this by forcing a fixed addition order in its kernels. Barry notes that the cure holds only on one GPU with those kernels. An H100 and an A100 ship different matrix-multiply kernels, and a CPU uses different instructions altogether, so three machines still give three answers.
His alternative is blunt. Integer addition is exact, so ordering cannot change the result, provided overflow is bounded in advance. Existing int8 inference does not get there, because it only simulates quantization: the matmuls run in int8 while softmax, RMSNorm, SwiGLU, and rotary embeddings fall back to floats. One float step anywhere lets reduction order leak back in.
DetLLM therefore keeps the whole path in integers, from token ids in to int32 logits out. The operators come from I-LLM, earlier work on integer-only transformers aimed at edge hardware and built for efficiency, not determinism. At 8 bits it skips fine-tuning and relies on a calibration pass plus two smoothing tricks that have closed-form solutions. Quantization happens per token using only that token’s numbers, so batch composition cannot touch them. The KV cache uses fixed scales calibrated offline, which keeps prefill and decode on the same bits.
The test is a strong one. Barry took the first 64 tokens of Sonnet 18, generated 512 more greedily, and hashed the logits at every step across nine configurations on an A100, an H100, an AMD EPYC, an Intel Xeon, and an Apple M1 Pro. The integer runs all returned the hash 64430dd985f8. The nine fp16 runs returned nine different hashes. His test suite checks exact equality of token ids and full logits, and all 79 tests pass on both GPU machines.
One finding complicates the pitch. In the fp16 column, only one run diverged in text; the other eight matched on all 512 tokens, because argmax ignores small perturbations. At step 457 the A100 saw ” introduces” lead ” is” by 0.03, while the M1 Pro hit an exact tie and the text forked. Barry’s reading is that float nondeterminism is usually latent, not absent, and it grows with output length.
Quality held up. WikiText2 perplexity came in at 20.72 against 20.95 for fp16, and C4 was 0.4% worse. Speed is the cost. Integer decode started at 0.8 tokens per second and reached 106 on an A100 at batch 1 after op fusion, CUDA graphs, and a fused Triton attention kernel. The fp16 baseline managed 29.3 there, though Barry flags that the comparison is uneven since the baseline had no compile step or CUDA graphs. Prefill still runs at about 0.13 times fp16.
The limits are plain. DetLLM covers one 0.6-billion-parameter model, and Barry lists larger models, other architectures, integer sampling, and phone or NPU hardware as open. The guarantee is also tied to one prepared artifact, because the float-to-int conversion is not itself reproducible.
Teams that need audit-grade replay of model outputs across vendors, such as regulated workloads or on-policy RL training, now have a concrete reference run to reproduce before deciding whether an int8 path is worth the kernel work.
Based on Nathan Barry’s post “Eliminating Nondeterminism in LLM Inference with Integer-Only Arithmetic” on nathan.rs, published November 12, 2026.
