Robots and self-driving cars have a training problem. The real world is the only place they can truly prove themselves, but it is also slow, expensive, and unforgiving: every edge case has to be found on a real road or a real factory floor, and every mistake has physical consequences. Simulation is the obvious answer, but until recently the simulations themselves were the bottleneck — hand-built, narrow, and never quite matching the messy reality they were supposed to stand in for.
On November 5, 2026, NVIDIA took a significant step toward fixing that. The company released four new open-weight checkpoints in its Cosmos 3 world-model family, all built on Cosmos 3 Nano, designed to generate interactive simulations for autonomous driving and robotics. The models can synthesize synchronized video from up to 11 camera views, matching LiDAR point clouds, and video of robots responding to their own actions — and they ship under a license that permits commercial use. For teams building physical AI, it is the most practical open release in the field this year.
What NVIDIA announced on November 5
The release, published in an announcement on the Hugging Face blog, adds four checkpoints to the Cosmos 3 family: Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual. All four are post-trained versions of Cosmos 3 Nano, each with 16 billion trainable parameters, and all four use the OpenMDW 1.1 license, which permits commercial and non-commercial use under its terms.
The four models split the work between two industries. Two of them serve autonomous driving: one generates synthetic driving data offline, the other streams driving observations in real time. The other two serve robotics: one turns a physics simulator’s controls into photorealistic video, and the other predicts what a robot will see next as it moves its arms. Together they cover the full loop of physical AI development — generate the data, run the simulation, train the policy, and test it again.
The release also includes a technical report, post-training recipes, inference notebooks, and agent skills for running each model, so teams can adapt the checkpoints to their own sensors and robots rather than treating them as black boxes. It is the largest expansion of the Cosmos 3 family since the platform’s main launch at GTC Taipei in May 2026.
Transfer-Auto: synthetic driving data across 11 camera views
The first checkpoint, Cosmos3-Nano-Transfer-Auto, is the offline workhorse. It is a bidirectional model built for generating synthetic driving data in bulk: given scene and traffic controls plus a text prompt, it produces synchronized video across as many as 11 camera views, including fisheye lenses, and it can generate matching LiDAR point clouds at the same time.
That synchronization is the point. For autonomous driving, appearance and geometry have to agree — a camera frame that shows a car where the LiDAR says there is empty space is worse than useless for training a perception system. Transfer-Auto uses per-camera World Scenario Map controls to specify the road layout, traffic motion, and signal states, while text prompts guide the appearance. Developers can turn the same drive from a sunny afternoon into a rainy night while keeping the road layout and vehicle trajectories identical, which is exactly the kind of variation that is hardest to collect in the real world.
The model supports 30 frames-per-second video at 480p or 720p, with clips of up to 400 frames, and NVIDIA’s reference run produces seven synchronized 720p videos and 99 LiDAR sweeps from a single scenario. It is also designed as a starting point: teams can post-train it on their own driving data to match their own camera rig and LiDAR configuration, then use the adapted model for offline data generation — or distill it into the streaming variant that follows.
Sim-Auto: streaming simulation that closes the loop
Offline data is only half the job. A driving policy also needs to be tested in a loop, where what it does changes what it sees next. That is what Cosmos3-Nano-Sim-Auto is for: the causally distilled streaming version of Transfer-Auto, it generates successive small chunks of synchronized camera video and LiDAR in just a few denoising steps, using the history it has already generated and updated driving controls.
The practical difference is speed and interactivity. Instead of rendering a whole clip and then evaluating it, Sim-Auto produces observations incrementally as a driving policy or a human driver updates the scenario, so the policy’s decisions feed back into the simulation in real time. Closed-loop interaction does require a simulation runtime and a renderer to update the scenario controls from driving inputs, but the payoff is long, interactive rollouts where a policy can be tested against the consequences of its own mistakes — safely, in simulation, before it ever touches a real road.
Sim-Transfer and Sim-Bimanual: robots that rehearse before they act
The two robotics checkpoints solve a different version of the same problem. Cosmos3-Nano-Sim-Transfer takes a physics simulator’s spatial controls — depth, segmentation, edges, and blur — plus a text prompt, and renders photorealistic RGB video from them. The simulator keeps computing the motion, contacts, and physical state; the model only supplies the visuals. That division of labour matters, because a policy trained on generated video needs the physics underneath it to be real, not hallucinated.
Cosmos3-Nano-Sim-Bimanual goes one step further: it predicts the next video observation directly from a robot’s actions, with no simulator in the loop at all. It supports interactive two-arm simulation for YAM and AgiBot robot embodiments, controlled by a learned policy or by a person driving it with joysticks. Start with an image, move the robot or the camera, and the model generates what the robot would see next — including switching between viewpoint changes and manipulation within a single control sequence.
This is the capability that connects most directly to the robots now entering real workplaces. A robot that can rehearse a manipulation task thousands of times in a generated world arrives at the factory already competent, instead of learning on the job. The same physical AI wave is showing up across the industry: Serve Robotics’ Physical AI Services turn fleet operations into a product for other robot builders, Embodied AI’s AIR humanoid learns factory jobs by watching workers, Cainiao’s embodied AI warehouse robots are training inside live logistics networks, and Agility’s Digit humanoid is already doing paid warehouse work. Cosmos 3 Nano gives all of them a cheaper place to practice.
Open weights, real hardware, honest limits
Open weights are only useful if teams can actually run the models, and NVIDIA has been specific about what that takes. The checkpoints require Linux, an NVIDIA GPU, Docker, and the NVIDIA Container Toolkit, and the published streaming setup was tested on a single GB200. NVIDIA’s own memory table puts Cosmos 3 Nano generation at 58 GiB of VRAM in BF16, or 44 GiB in FP8 — workstation-class hardware, not a laptop, but well within reach of a serious robotics lab.
The release is also honest about what it does not yet prove. NVIDIA says plainly that more work is needed to measure where the models faithfully support policy learning and evaluation, and to understand their limits. That candour is welcome: a world model that looks convincing is not the same as one that teaches a policy the right lessons, and teams should evaluate each checkpoint on their own tasks before trusting it in production. The open post-training recipes and the four-stage training pipeline — bidirectional adaptation, teacher forcing, consistency distillation down to four denoising steps, and self-forcing on generated history — exist precisely so teams can do that evaluation and improve the models themselves.
The surrounding platform is growing too. NVIDIA’s technical blog walks through the Cosmos Framework’s agent skills for training, reinforcement learning, and deployment, plus Cosmos NIM microservices for serving the models as production APIs, with measured speedups of roughly two to three times on H200 hardware. The platform now also reaches beyond driving and factory robots: the same family includes surgical simulation models that generate video from robot actions to evaluate policies without building a physics model of tissue. For the full technical detail, read NVIDIA’s technical blog and the NVIDIA Cosmos platform page.
Why this matters for physical AI
The bigger picture is a shift in where physical AI gets built. Until now, the ability to simulate the world well enough to train a robot was mostly locked inside large companies with proprietary data and private simulators. An open-weight release with a commercial-use license, published recipes, and runnable notebooks lowers that barrier for everyone else — startups, university labs, and the smaller robotics companies that cannot afford to collect millions of miles of driving data or years of factory footage.
It also changes the economics of robot training. Real-world data collection is the most expensive part of building a physical AI system, and it is the part that scales worst: every new environment, weather condition, or robot embodiment starts the collection over. Generated simulation does not replace real data, but it multiplies it, covering the rare and dangerous cases that real fleets encounter too infrequently to learn from, and letting policies fail safely thousands of times before they are trusted with real hardware.
That is why the timing matters. Humanoid robots are moving from demos to paid deployments, robot fleets are becoming products, and logistics networks are opening up as training grounds. The industry now has the robots; what it has been missing is enough world for them to practice in. Cosmos 3 Nano does not solve that problem on its own — the fidelity questions NVIDIA flags are real, and the hardware bar is high. But it puts a serious, open, commercially usable simulator into the hands of every team building physical AI, and that is the kind of release that compounds. The robots that learn fastest in the next year may be the ones that spent the most time in worlds that never existed.
