Qiyuan Releases PrimeSim World Model, Tops Two Robotics Benchmarks

30,000 hours of training data, compatibility with 22 types of robot embodiments, and cross-embodiment few-shot generalization capabilities.

Published by Synced (Machine Intelligence)

Training robot control policies relies heavily on simulation environments, but traditional manually built simulators have extremely high barriers to large-scale deployment.

Today, the Qiyuan Embodied Intelligence Research Center officially released PrimeSim, a world model simulator designed for robotic manipulation tasks. Based on pre-trained video generation models and refined through two stages—mid-training and post-training—it possesses interactive rollout capabilities that can replace traditional simulators.

On the EWMBench benchmark, PrimeSim achieved a comprehensive score of 6.09, ranking first; in the five evaluation dimensions of WorldSimProbe, it secured the top position in three categories.

PrimeSim generated demos

Why Robots Need "Data-Learned Simulators"

An ideal robot simulator should meet several core criteria: the ability to parallelize tens of thousands of trajectory rollouts, support precise scene resets without hardware wear, require no extensive engineering customization for different scenarios, and minimize the reality gap (sim-to-real gap).

However, using only video generation models for this task is insufficient if they merely achieve "visual realism." As a simulation foundation, it must simultaneously satisfy three constraints:

1. Strict adherence to action commands output by the policy;
2. Geometric and semantic consistency across multiple camera views;
3. State coherence maintained during long-term rollouts that far exceed the training context window.

The team believes that most existing general-purpose world models can only satisfy one or two of these requirements, making it difficult to balance all three.

Technical Path: Mid-Training First, Then Post-Training

PrimeSim adopts a two-stage transformation paradigm, gradually converting native bidirectional video generation models into interactive, autoregressive rolling world model simulators.

Stage 1: Mid-Training | From Video Generation Model to Robot World Model

Model training is based on 30,110 hours of curated robot demonstration data, covering 22 heterogeneous robot platforms including humanoids, dual-arm robots, single-arm robots, and handheld UMI devices. The dataset mixes real-world collected data with traditional simulation data; humanoid robot samples account for 54.8%, and real-world data accounts for 84%. The core design includes three key aspects:

• Unified Action Representation: Control signals for the head, torso, waist, end-effector, gripper, and chassis are encoded into a single action vector, enabling cross-embodiment compatibility;
• Frame-Aligned Dual-Path Conditional Injection: Combines temporal alignment attention mechanisms with direct action injection to strongly bind control commands with generated motion;
• View-Aware Interleaved Attention: Introduces explicit camera identifiers to facilitate cross-view information interaction, ensuring multi-view consistency and preventing collapse between different view frames.

Stage 2: Post-Training | High-Speed Generation + Long-Term Temporal Memory

The challenge of autoregressive simulation lies in the fact that the model must continue rolling out based on its own previous, potentially imprecise predictions, whereas native video generation models are trained only on fixed-length ground-truth contexts. The team specifically designed three mechanisms to address this:

• High-Speed Generation Pipeline: Block-wise causal adaptation + four-step consistency distillation supports continuous simulation over long sequences;
• Self-Conditioning Stability: Uses a frozen bidirectional teacher model as a reference to conduct Self-Rollout training for the world model simulator and asymmetric distribution matching, mitigating error accumulation;
• Hierarchical Temporal Memory: Stores initial scenes, sparse long-range history, and dense recent motion hierarchically within a bounded KV Cache, supporting ultra-long rollouts.

This approach ultimately transforms the bidirectional generation model into a causal, Self-Rollout world model simulator: it can maintain stable rollouts even beyond the fixed training context window, supporting continuous rollouts of more than two minutes at maximum.

On the RoboTwin 2.0 held-out set, using a single A100 GPU, the causal autoregressive diffusion baseline with 50 denoising steps achieves only 1.12 FPS and a PSNR of 26.15. By compressing the denoising steps to 4 via Consistency Distillation (CD), the speed increases to 10.04 FPS, while the PSNR drops to 24.40. With the addition of Distribution Matching Distillation (DMD), maintaining 4 denoising steps restores the PSNR to 24.87 and improves LPIPS from 0.193 to 0.176, resulting in an overall ~9x speedup compared to the 50-step teacher model.

Experimental Results

EWMBench: Ranked First in Overall Score

PrimeSim achieved an overall score of 6.09 on EWMBench, surpassing GE-Sim (LTX, 5.82) and GE-Sim (COSMOS2, 5.79), as well as leading Thinker-WM, EnerVerse-FT, Qwen-RobotWorld, and Sora2. In individual metrics, PrimeSim ranked first in HSD (1.000), nDTW (1.000), BLEU (0.570), CLIP (0.954), and Logics (1.000).

WorldSimProbe: Three Wins Across Five Metrics

WorldSimProbe is designed based on LIBERO with five evaluation suites, deliberately isolating visual aesthetics and task success rates to focus on the simulator's dynamic fidelity. PrimeSim topped three categories: Global Trajectory Coverage (T2, 67.9), Action-Source Behavior Preservation (T3, 72.3), and Interaction Dynamics (T5, 37.1).

As a Policy Simulation Base: Highly Aligned with Traditional Simulators

The experiment selected 8 OpenVLA-OFT policies. The baseline simulators showed policy success rates ranging from 29.3% to 85.0%. Under a setting with 300 shared initial states, PrimeSim demonstrated a correlation coefficient of r = 0.99 with the native LIBERO simulator regarding policy success rates, with an MMRV of 0.000 and a Mean Absolute Error |Δ| of 2.8. For comparison, Wan2.2-FT yielded corresponding metrics of r = 0.91, MMRV = 0.041, and |Δ| = 8.3.

The team further conducted GRPO reinforcement learning directly within PrimeSim for OpenVLA-OFT, sampling 512 episodes per round for a total of 200 update rounds. The policy's success rate on the native simulator improved by 11.5%, while evaluation metrics within PrimeSim rose by 11.8%. This demonstrates that policy gains obtained within this neural simulator can effectively transfer to traditional physical simulation environments.

Counterfactual action following demos

Few-Shot Embodiment Transfer: New Embodiments Require Only 100–200 Fine-Tuning Steps

Mid-training endows the model with strong cross-embodiment transfer capabilities. Adapting to xArm 7, ViperX 300, and UMI grippers requires only 200, 100, and 200 gradient update steps, respectively; whereas models trained solely on general pretraining fail to produce usable results even after 2000 update steps. In the AgileX Piper XVLA scenario, where cloth, camera poses, and scenes were absent from the training set: zero-shot inference shows rapid drift in dense-contact cloth deformation; however, on a single A100 GPU, after less than 3 minutes and 200 fine-tuning steps, the cloth deformation follows action commands, and multi-view state consistency is maintained.

PrimeSim unseen embodiment few-shot demos. Left side is PrimeSim, right side is Wan2.2 5B

Unseen camera and environment zero-shot demos

Inference Performance: Reaching 10.12 FPS on a Single A100

With a single A100-80GB GPU, using three views, a single image resolution of 288×384, predicting 32 future frames per pass, and a 4-step denoising configuration, PrimeSim achieves 10.12 TempFPS and 30.35 ViewFPS. Its throughput is approximately 4.5 times that of GE-Sim 2.0 and 11.6 times that of Ctrl-World.

Ablation Studies: Each Module Design Yields Clear Benefits

Ablation studies on the multi-view mechanism show: frame concatenation alone yields a PSNR of only 16.30; adding view token conditioning improves it to 23.78; incorporating intra-view interleaved attention reaches 24.56, reducing LPIPS from 0.58 to 0.15.

Summary of Work

Based on experiments, the team drew four core conclusions:

1. This is a reproducible engineering recipe, not just a single-point model: The progressive "mid-training + post-training" pipeline can transform general video generation models into interactive robotic world model simulators;
2. Action fidelity can be learned through data scale: Unified action representation combined with dual-path conditional injection achieves strong binding between control signals and motion generation across 22 heterogeneous embodiments;
3. World models have the potential to replace traditional simulators: High correlation (0.99) with native simulator policy success rates, and benefits from reinforcement learning conducted within the model are transferable;
4. Balancing accuracy and engineering practicality: 10.1 FPS inference speed, long-horizon rollout exceeding two minutes, and new embodiments requiring only hundreds of fine-tuning steps.

According to Jiqizhixin, PrimeSim's weights, inference code, and training code will soon be open-sourced.

© THE END

Please contact this official account for authorization before reprinting.

For submissions or media inquiries: [email protected]

Open in WeChat