Nvidia Open-Sources GR00T 2 Robot Foundation Model

henry reporting from the Non-Orthodox Temple QuantumBit | WeChat Official Account QbitAI

Everyone stand up! After a year and a half, NVIDIA's new-generation robot foundation model is finally here!

Just now, at this year's CoRL conference held in Austin, NVIDIA GEAR leaders Yu-Ke Zhu and Jim Fan unveiled the new-generation world action model—

GR00T 2.

In the official 1.5-minute demo, GR00T 2 can cut labels with scissors, install graphics cards, strike matches, slice bananas to make breakfast, wipe up coffee, tidy up tables, and sell popcorn.

Watching it all, it seems almost omnipotent. From fine bimanual manipulation to whole-body locomotion, the model's task range covers nearly every direction of robot capability.

Moreover, according to Yu-Ke Zhu, GR00T 2's model, data, simulation environment, and data collection hardware will also be open-sourced progressively.

Currently, 100,000 hours of human data have already been released first.

Let each of us say "NVIDIA is awesome"—that's not too much to ask, right?

GR00T 2: Scaling Generalization, Dexterity, and Adaptability of Robot Foundation Models

As a world action model, GR00T 2 can simultaneously predict future frames and robot actions. During training, it learns "how the world will change" and "how the robot should act" together.

At the launch event, Jim Fan positioned it as a System 1 (fast system) physical skill model that reacts quickly enough, can run locally, and is both easy to teach through demonstrations and easy to guide by reasoning models.

And inside GR00T 2, this is what Yu-Ke Zhu called X-Adapt cross-embodiment adaptation on Twitter.

Simply put, in the past, teaching a robot a new task often required collecting a fresh batch of robot demonstration data specific to its body form and control method.

X-Adapt attempts to break through this limitation: skills demonstrated by humans or handheld devices can be directly transferred to robots that have never practiced the task.

Specifically, NVIDIA demonstrated three pathways.

1. Transferring from human demonstrations to robots.

Researchers wore motion capture equipment and demonstrated how to place and wet filter paper. Subsequently, the dual five-fingered hand Dexter could complete the same task. Throughout the process, no robot teleoperation demonstrations of this task were collected. Based solely on human demonstrations, Dexter achieved a 65% success rate across 20 trials.

2. Transferring from handheld grippers to robotic arms.

Researchers used a handheld UMI gripper to demonstrate how to assemble a toy dump truck. After the model learned it, the skill was transferred to a YAM robotic arm. Although the UMI and YAM grippers have different forms, YAM could still complete assembly tasks lasting up to about four minutes end-to-end, with a 60% success rate. The same transfer pathway was also used for conveyor belt sorting and throwing, disassembling operation panels, and other tasks.

3. Transferring from wearable exoskeletons to robot dexterous hands.

Researchers wore the DexUMI hand exoskeleton to demonstrate making banana yogurt and scooping cereal. After fine-tuning the model with this data, the robot could take over and complete the breakfast task, again without collecting additional robot teleoperation demonstrations for this task.

In summary, these three pathways point to the same change: the same skill does not always need to be re-demonstrated on every kind of robot.

Of course, cross-embodiment transfer does not mean robot data is completely unnecessary. In the human motion capture and UMI pathways, the target robot still provides task-agnostic data to familiarize the model with its own body structure, motion patterns, and control space. Demonstrations for new tasks, meanwhile, can come from human operations that are easier to collect.

This is also the direction emphasized in the GR00T 2 technical report: scaling the generalization, dexterity, and adaptability of robot foundation models.

Beyond transfer, GR00T 2 also demonstrated the ability to improve execution precision through a small number of robot demonstrations.

In another Dexter-50 evaluation, the model's zero-shot average success rate was 40%; after providing one demonstration per task, the success rate rose to 71%; after providing five demonstrations, it reached 95%.

More critically, these five demonstrations totaled less than three minutes. Then, only about 30 minutes of LoRA fine-tuning on a single RTX 5090 was needed to obtain a model adapted to the corresponding task.

In terms of practical invocation, NVIDIA also kept up with the trend and opened the WAM API.

Through the API, reasoning models can handle task decomposition and check execution progress, then hand specific physical skills to GR00T 2 for execution.

In other words, GR00T 2 can not only complete skills on its own, but also serve as a physical execution module invoked by upper-layer agents.

So the question is: NVIDIA, why are you so awesome?

How Do 400,000 Hours of Human Video Become a Robot's Hands-On Ability?

Let's take them one by one. Behind X-Adapt, first of all, is a data pyramid. GR00T 2's pretraining mixed three types of data:

402,730 hours of human first-person video;

10,184 hours of handheld UMI demonstrations, covering 2,897 tasks;

23,500 hours of robot teleoperation data.

It is not hard to see that the scale of human video far exceeds the other two categories. This means robot learning materials are no longer limited to action trajectories collected through manual teleoperation, but further extended to a large amount of real human activity:

how people pick up tools, move objects, complete assembly, and how the surrounding scene changes after an action.

UMI and wearable devices supplement more direct manipulation trajectories, while robot teleoperation data helps the model learn the motion and control methods corresponding to different bodies.

To verify whether human video can actually help robots do work, the team also conducted a dedicated set of scale experiments.

They trained models with data containing no human video, 1%, 10%, and all human video, respectively, and then evaluated them on 72 Dexter tasks.

The results showed that as human video data increased, the average task progress on real robots rose from 50% to 70%.

This set of experiments gives a fairly direct answer: large-scale human experience not only enriches a model's visual knowledge, but can also translate into concrete execution progress for robots on real tasks.

To ingest this data, on the model side, GR00T2 receives language instructions, multi-view camera frames, and robot states, and jointly predicts future video and action sequences through a single diffusion Transformer.

Finally, around GR00T2, NVIDIA is also connecting data collection, processing, training, simulation validation, and real-world deployment into a complete toolchain.

On the collection side, there are devices such as UMI and DexUMI. They let researchers directly record human operations, especially dexterous motions that are difficult to complete naturally with traditional teleoperation.

On the data side, there are Human-1M and Flora. The former provides first-person human videos, with an initial 100,000-hour subset already public; the latter supports data curation, descriptive annotation, and automatic evaluation.

On the simulation side, there is SimFoundry. It can reconstruct task environments from real scenes for policy testing, data generation, and skill development.

On the execution side, different models and components divide the work.

ASPIRE organizes robot models and code skills to support task execution and error recovery. For humanoid robots, Turbo uses a lighter network and a more compact video representation to reduce prediction overhead.

On the onboard Jetson Thor of G1, Turbo takes 124 milliseconds per prediction; SONIC runs on another Jetson Orin, performing whole-body control at 50 Hz.

The former generates motion plans, and the latter turns the plans into continuous body movements, allowing the robot to maintain coordination when reaching, bending, and carrying.

In this way, after developers get GR00T2, they can not only call the model itself, but also use the accompanying toolchain to collect their own data, adapt to their own tasks, validate skills in simulation, and finally deploy them to real robots.

This is also what makes NVIDIA's release more noteworthy this time: what it opens up is no longer just a robot model, but a complete development system from data collection and model training to skill invocation and actual execution.

One more thing

In his on-site talk at CoRL, Jim Fan said:

"Astra didn't shatter our dreams; it broke down our walls."

"This is not a winter, this is the summer of robotics."

That said, the newly released Claude Fable 5.2 today also performs in a way on RoboDojo that somewhat seems to be teaching robotics a lesson.

In the RoboDojo Sim dual-arm test published by Anthropic, using the same runtime framework and the highest available inference strength for each model, Fable 5.2 scored 64.9%, while GPT-6 Astra scored 29.8%.

How about next time at "See You Later," the large model team and the embodied model team also sign up?

References [1]https://research.nvidia.com/labs/gear/gr00t-2/ [2]https://research.nvidia.com/labs/gear/gr00t-2/assets/gr00t2_whitepaper.pdf