New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

From the source

NVIDIA Cosmos world model development platform adds four new open-weight checkpoints to its Cosmos 3 model family for generating interactive world simulation and multi-sensor multi-view driving scenes 🌦️ Cosmos3-Nano-Transfer-Auto : Generate multi-view driving data with varied weather, lighting, and time of day.

🚗 Cosmos3-Nano-Sim-Auto : Stream synchronized camera and LiDAR observations for driving simulation. 🎨 Cosmos3-Nano-Sim-Transfer : Turn simulator controls and a text prompt into photorealistic video. 🤖 Cosmos3-Nano-Sim-Bimanual : Predict video from robot or camera actions.

All four models are post-trained examples of Cosmos 3 Nano and have 16B trainable parameters. The models use OpenMDW1.1, permitting commercial and non-commercial use under its terms. Download the checkpoints · Read the technical report . Watch the demos Why do you need these checkpoints?

🦾 Robotics needs environments that can answer this across changing scenes and viewpoints. These checkpoints offer complementary ways to build them: Sim-Bimanual predicts visual consequences from actions, while Sim-Transfer turns a physics simulator’s controls into generated observations. …

huggingface.co