Nvidia's next-gen world-action model that jointly predicts future frames and robot actions with cross-embodiment transfer.
Nvidia has introduced GR00T 2, a robotics foundation model that departs from directly mapping camera images and language commands to motor actions. Instead, the model watches humans in video and internally imagines itself performing the motion as a robot before producing control signals, aiming to sharply reduce costly and slow teleoperation data collection.
Nvidia unveiled GR00T 2, a new world-action model that jointly predicts future frames and robot actions, at CoRL. It enables cross-embodiment skill transfer from human demonstrations, letting the Dexter robot hit 65% success on a filter-paper task with no robot teleoperation data. Nvidia says the model, data, simulation environment and capture hardware will be open-sourced, with 100,000 hours of human data already released.