NVIDIA has released a set of Cosmos tools that generate matching camera video and LiDAR readings for simulated driving, according to a post on the company’s developer blog. LiDAR is the laser sensor many self-driving cars use to measure distance. The same release covers robot arms and surgical robots, and adds a packaged way to run the models on NVIDIA’s own GPUs.

The core idea is the “world model,” an AI that predicts what a physical scene looks like as it changes. NVIDIA’s Cosmos platform bundles these models with developer tools. The blog post splits the work in two: adapting a model to your own sensors and data, then getting it into a real application without it falling over.

For driving, NVIDIA describes two checkpoints (saved model versions) built on Cosmos 3 Nano. The first, Transfer-Auto, takes a text prompt plus a map-style description of lanes, signs, and objects, then renders a whole multi-camera clip at once. Change the prompt and a daytime drive becomes a rainy night on the same road layout with the same vehicle paths. That is useful for testing perception software against weather it rarely sees in recorded data. NVIDIA’s reference run on four GPUs produces seven 720p videos of 297 frames each, plus 99 LiDAR sweeps.

The second checkpoint, Sim-Auto, is a faster, trimmed-down version that generates frames as it goes. A driving policy or a human at the controls changes the scene, and the generated camera and LiDAR views feed back into the policy. Developers call this closed-loop testing, and it is where simulators usually earn their keep or fall apart.

Robots get their own variants. Sim-Transfer takes depth footage plus a text prompt and returns photo-style video with new lighting and backgrounds while keeping the motion intact. Sim-Bimanual predicts the next view from the scene and the robot’s actions, for two-armed machines such as the YAM and AgiBot. For surgery, NVIDIA offers Cosmos3-H-Embodied-Nano and a real-time version called H-Dreams. NVIDIA’s pitch is that this lets teams evaluate surgical policies without hand-building a physics model of tissue.

On the training side, NVIDIA says its Cosmos Framework now includes reinforcement learning methods and “agent skills,” instruction files that let a coding agent run setup, post-training, and troubleshooting. The company also says the framework will extend beyond Cosmos 3 to models such as Alpamayo, GR00T, and pi0.5.

Deployment runs through Cosmos NIM, NVIDIA’s prebuilt containers for serving models. NVIDIA says speculative decoding gives average speedups of roughly 2 to 3 times on its H200 chip, and that live-video analysis can raise alerts with 10 times lower latency. Memory needs are published: the Nano generation model needs 58 GiB of GPU memory at standard precision, 44 GiB at FP8, and 31 GiB with offloading. The larger Super model needs 150 GiB at standard precision.

Every performance figure in the post comes from NVIDIA, and the post offers no independent comparison. It also does not say how closely the generated LiDAR matches what a physical sensor would record. That gap matters, because a car trained on convincing but wrong point clouds learns the wrong lessons.

Teams building perception or robot policies can run the released checkpoints on Hopper or Blackwell GPUs now. NVIDIA is hosting a livestream on the framework on 5 November, and anyone weighing it should bring a real-versus-generated LiDAR comparison from their own rig to it.

Reported by the NVIDIA Developer Blog on 2 November 2026.