llm-d, an open-source project that runs AI models across a cluster managed by Kubernetes (the standard system for orchestrating servers), is pitching itself as the shortcut from a lone test machine to production serving. Its website says the stack works with vLLM, SGLang, and other inference engines, on GPUs, Google TPUs, Intel XPUs, CPUs, and what the project calls emerging NPUs.
The offer is a set of “Well-Lit Paths.” The project describes them as deployment recipes that have been tested against common production setups, meant as starting points teams adapt to their own models, hardware, and workloads. The site lists version 0.8.1 and says the community welcomes everyone, from newcomers to people debugging complex deployments.
The performance numbers on the page, including 3x higher output throughput and a 40% cut in time to first token and inter-token latency in Google Vertex, are the project’s own claims. The page gives no baselines, so they cannot be weighed yet.
Platform teams already running Kubernetes on mixed hardware have a cheap experiment here: stand up a test cluster and let your own benchmarks decide.
Drawn from the llm-d project website (llm-d.ai), read on 22 October 2026.
