Autoscaling a large language model is expensive mainly because of what happens before the first token: every new replica has to pull the same multi-hundred-gigabyte weights from scratch before it can serve a request. NVIDIA says its new ModelExpress system fixes this by letting a fresh replica copy those weights straight from another replica’s GPU memory. In NVIDIA’s own benchmark, that change cut the cold-start time for DeepSeek-V4 Pro, an open model with roughly 806 gigabytes of weights, from 8 minutes to 1 minute 44 seconds, a reduction of roughly 78 percent.
The mechanism is a priority check. Before loading anything, ModelExpress asks whether a compatible copy of the weights already sits somewhere in the cluster. If one does, the new replica pulls it directly using P2P RDMA (remote direct memory access, a method that lets one GPU read another GPU’s memory without routing data through a CPU or disk), moved over NVIDIA’s Inference Xfer Library, known as NIXL. Only the first replica in a deployment has to bootstrap from object storage or local disk. Every replica after that can fan out GPU to GPU.
ModelExpress also targets a cost that shows up on the storage side once a fleet scales out. NVIDIA says that if ten replicas of the 806 gigabyte DeepSeek-V4 Pro model start at once, the cluster would otherwise need to pull roughly 8 tebibytes of duplicate data across the same network link. A cache service inside ModelExpress coordinates a single external download and lets every other replica reuse it, so the fleet pays that download cost once no matter how many replicas spin up.
Weight loading is not the only tax on startup time. Once weights are in memory, the serving engine still has to compile and tune kernels for that specific model, precision, and GPU, a step NVIDIA says can take several minutes on its own and becomes the dominant cost once weight transfer stops being the bottleneck. ModelExpress extends the same peer-to-peer transfer to those compiled kernel caches, so a new replica can inherit a warmed-up engine instead of repeating the compilation.
The 78 percent figure is NVIDIA’s own, measured on a test node built from eight NVIDIA B200 GPUs with ConnectX-7 networking, running the vLLM inference engine, version 0.23.0. NVIDIA has not published comparable numbers for smaller GPU counts, older network fabric, or models substantially larger than DeepSeek-V4 Pro, so operators without that specific interconnect should treat the gain as a ceiling rather than a guarantee.
NVIDIA is also extending the peer-to-peer approach to reinforcement learning post-training, where a trainer updates weights every step and inference workers must pick up the new version before generating the next round of rollouts. The company says teams at Fireworks, Cursor, and Cognition are already testing related weight-diffing techniques in production RL runs, though ModelExpress’s own receiver-driven version of that transfer remains an early integration rather than a shipped default.
For any team running elastic inference on frontier-scale open models, cold-start time is what makes autoscaling expensive rather than free, since every scale-out event repeats the same multi-minute load. A serving fleet that already sits on NVIDIA GPUs with RDMA-capable networking has a concrete new lever to test against its current provisioning buffer before committing next quarter’s capacity budget to it.
NVIDIA published these ModelExpress benchmarks and technical details on its developer blog on July 24, 2026.
