Nvidia rolled out NeMo Switchyard this week, an open-source library that decides, step by step, which AI model should handle the next move in an agent’s workflow instead of locking the whole task to one model from the start. The company is pitching it as a fix for a specific enterprise headache: paying frontier prices for agent work that is mostly routine, or building custom routing logic that breaks every time a workflow changes.

Nvidia’s central claim is about cost. Paired with a compatible model pool, the company says the setup holds frontier-level task completion at close to a third of the price of running Anthropic’s Opus 4.8 for every step, based on a benchmark comparison Nvidia designed and ran itself. No outside lab has verified that ratio, and the comparison point, Opus 4.8, is one Nvidia chose rather than a standardized industry baseline.

The router cannot produce that savings alone. Switchyard needs somewhere cheap and capable to send routine work, and Nvidia shipped that piece alongside it: Nemotron 3.5 Lightning, an open, 30-billion-parameter mixture-of-experts model Nvidia built to handle frequent, narrowly scoped agent tasks. Inside a Switchyard pool, Lightning functions as one available destination for routine steps before harder ones escalate to a frontier model. Nvidia is selling the pairing, not either piece in isolation, as the actual product.

Kari Briski, Nvidia’s vice president of generative AI, said the router adjusts to an agent’s changing state rather than sticking to one model per task type. She described options ranging from a random router, which she called “not that great,” to a classifier route or an agent-state route that picks whichever model fits a given step. The router also factors in a metric Nvidia calls model verbosity: an estimate of the token count a model will likely need to finish a step, used to steer cheap, routine work away from chattier models before the call goes out.

Switchyard is not trying to displace existing routing infrastructure. It plugs into agent frameworks including LangChain, Cognition, and Nous Research, and into LLM gateways including LiteLLM, Kong, and OpenRouter, the last of which already runs a rival routing product called Not Diamond. RouteLLM, an open-source router built by UC Berkeley and LMSYS researchers, is another existing option. Neither Not Diamond nor RouteLLM ships its own model, which is the gap Nvidia is betting it can exploit by controlling the router and a purpose-built cheap model under one license.

Early partner figures, also supplied by Nvidia, support the general pattern without matching its headline multiplier:

Each figure comes from a different task set measured by a different company using its own baseline, not a shared, standardized test. That matters because Nvidia’s own third-of-the-cost figure carries the same limitation: it is Nvidia’s benchmark, Nvidia’s chosen comparison model, and Nvidia’s own report of the result.

The bigger change is procedural rather than architectural. Teams that once pointed an entire agent pipeline at a single default model now face pressure to route each step by live signals, including how far a task has progressed and what finishing it will cost. That approach only pays off if the routing layer stays current without becoming a new maintenance job every time a workflow shifts, which is the gap Nvidia says Switchyard is meant to close.

Teams currently running always-on agents against one frontier model by default should benchmark a Switchyard pool against their existing setup before their next contract renewal. Even discounting Nvidia’s own multiplier, the partner-reported figures point to real, double-digit cost reductions available from routing alone.

Sean Michael Kerner reported this story for VentureBeat, published August 12, 2026.