Fysiverse-UniWAM Unifies Navigation and Manipulation Without Large-Scale Robot Pretraining
Fysics AI and Fudan University introduced Fysiverse-UniWAM, a unified model for autonomous navigation, mobile manipulation, and dexterous operations. By leveraging video world models instead of large-scale robot pretraining and constructing the MAP-Data dataset, it achieved top performance in 24 real-world tests, reducing average position error by 30.1% compared to baselines.
Full Report
Contents4 sections
One Model Handles Navigation, Docking, and Grasping: New Progress in Domestic Embodied AI
Edited by | Chen Chen
Robots are finally not just knowing where to go, but also where to stop, which direction to face the target, and even being able to start working immediately after stabilizing.
This sounds obvious, but for robots, it has been a long-standing challenge.
For example, consider asking a robot to clear cups from a table. Being able to reach the table does not mean it can pick up the cup. If the chassis stops slightly too far away, the robotic arm may not be able to reach; if the orientation is slightly off, subsequent grasping may be affected.
How can we enable robots not only to reach their destination but also to dock precisely, grasp objects, and perform more complex dexterous operations?
Recently, a domestic team proposed an interesting new approach.
Fysics AI, in collaboration with the Cognitive Intelligence Technology Lab (CITLab) at Fudan University, introduced Fysiverse-UniWAM, a unified world-action model. It attempts to integrate three originally relatively independent capabilities—autonomous navigation, mobile manipulation, and fixed-station dexterous manipulation—into a single model.
More notably, this model does not follow the traditional route of starting with large-scale robot pre-training.
The team started from a pre-trained video world model, achieved joint learning of navigation and manipulation through a shared backbone network, and proposed MAP (Manipulation Anchor Pose). They constructed the MAP-Data dataset, comprising over 1.5 million trajectories and approximately 7,500 hours of data.
With a different technical route, how does the actual performance hold up?
The results were somewhat surprising.
In the MAP-Bench benchmark tests, Fysiverse-UniWAM achieved an average position error of only 9.85 cm and an average orientation error of 2.30°, representing reductions of 30.1% and 44.0% respectively compared to the strongest external baseline.
Even more impressive, the team directly deployed the model on four types of robot platforms, conducting 24 real-world tests covering autonomous navigation, mobile manipulation, and fixed-station operations.
Among these, the model led in position error across all 13 navigation tasks and achieved the highest success rate in all 4 mobile manipulation tasks. In complex fixed-station operations, its overall performance approached that of π₀.₅, a model trained via large-scale robot pre-training.
Without relying on large-scale robot pre-training, how did a single model achieve such results?
24 Real-World Tests: How Capable Is This Single Model?
Let’s skip the technical principles for now and look directly at the real-world results.
Autonomous Navigation: Not Just Reaching, But Docking Precisely
Researchers moved the robot to positions far from the target, changing the starting point and orientation. The robot was able to autonomously plan an approach path based on the current environment, adjusting the chassis position and direction to ultimately dock precisely.
Even when switching targets, scenarios, or robot platforms, the model could still complete the corresponding navigation tasks.
This is not just a demo effect.
In 13 MAP navigation real-world tests, UniWAM ranked first in position error across all tasks and achieved the best orientation error in 11 of them.
The significance of precise docking lies in creating suitable conditions for subsequent robotic arm operations.
Mobile Manipulation: Getting There and Finishing the Job
In mobile manipulation tasks, the robot needs to autonomously approach the target, adjust the chassis, and then control the robotic arm to complete grasping and placement.
For example, the entire process—from moving to the side of the table, reaching out to grab a cup, and storing it—is completed autonomously by the model.
When switching to color block classification, the robot needs to identify the target color, coordinate the chassis and robotic arm, and place the blocks into the corresponding trays.
These tasks truly test the integration capability between navigation and manipulation, rather than just performing individual actions in isolation.
Table 7 in the technical report provides item-by-item evaluation results for 4 real-world mobile manipulation tasks. UniWAM-Scaled achieved the highest success rate in all 4 mobile manipulation tasks, with an average success rate of 80.0%, leading the current strongest robot pre-training baseline, π₀.₅, by 17.5 percentage points.
Item-by-item evaluation results for 4 mobile manipulation tasks in Table 7 of the technical report
Dexterous Manipulation: Universal Grasping, Dual-Arm Three-Layer Cup Stacking, High-Precision Tube Insertion, Long-Horizon Coffee Making
If navigation and mobile manipulation have already demonstrated the model's comprehensive capabilities, the following dexterous manipulation tasks are even more eye-catching.
First is a one-shot interactive demonstration of "Pick Up Anything."
Facing objects of varying shapes, sizes, and orientations, the robot can flexibly adjust its grasping actions based on real-time visual observations, completing interactive grasps of different targets consecutively.
As a human hand moves to different positions in 3D space, the robotic arm continuously perceives the hand's position and adjusts its delivery action accordingly, achieving precise object handover. Wherever the human hand reaches, the robotic arm delivers the object there.
Higher-difficulty operations are also handled effortlessly.
In the dual-arm three-layer cup stacking task, the robot needs to coordinate both arms, precisely grasp, and place cups layer by layer, ultimately building a three-tier cup tower.
In the high-precision test tube insertion task, the robot must control the position and pose of the robotic arm end-effector to complete fine alignment and insertion, placing higher demands on spatial precision.
In the long-horizon coffee making task, the robot needs to consecutively complete multiple operational steps, posing greater challenges for long-sequence task planning, action continuity, and execution stability.
From generalized grasping and dual-arm coordination to fine insertion, Fysiverse-UniWAM demonstrates rich dexterous manipulation capabilities.
In 7 fixed-station real-world evaluations, the model surpassed π₀.₅ in 3 tasks, achieving an overall success rate of 68.9%, trailing π₀.₅ by only 2.5 percentage points.
Success rate comparison for 7 fixed-station operation tasks in Table 7 of the technical report
Looking at the combined 24 real-world tasks, Table 6 in the technical report summarizes the overall performance across the three categories: UniWAM-Scaled achieved the best docking precision in both MAP navigation and mobile manipulation, ranked first in mobile manipulation success rate, and ranked second in fixed-station operation success rate.
Table 6 of the technical report presents the overall evaluation results for 24 real-world tasks.
How can a unified model, which has not undergone large-scale robot pre-training, achieve state-of-the-art performance in navigation and mobile manipulation while also handling such a wide variety of dexterous manipulation tasks? What are the underlying technical secrets?
Without relying on large-scale robot pre-training, what path did UniWAM take?
In recent years, large-scale robot pre-training—represented by models such as π₀.₅ and GR00T—has become a key technical approach in embodied AI.
However, obtaining real-world robot data is not easy. In particular, complete mobile manipulation trajectories require continuous collection of the entire process from movement and docking to arm execution, which entails significant cost and complexity.
More importantly, there are distinct differences between navigation and manipulation: the former focuses on environmental layout and chassis motion, while the latter emphasizes close-range object interaction and fine-grained robotic arm control.
So, is it possible to enable navigation and manipulation to learn jointly within a single model without relying on massive amounts of complete robot operation trajectories?
To address this, the team decomposed the problem into two levels.
The first question: How does the robot know where the appropriate navigation endpoint for manipulation is?
The second question: Given that navigation and manipulation differ in data types, action spaces, and control methods, how can they be learned jointly within a single model?
The corresponding solutions are MAP and Mixed-Stream.
Challenges in mobile manipulation and the Fysiverse-UniWAM solution.
MAP: Making the Navigation Endpoint the Starting Point for Manipulation
Traditional semantic navigation typically only requires the robot to reach the vicinity of a target. However, for mobile manipulation, simply "arriving" is far from sufficient.
The robot must also determine where the chassis should stop and in which orientation it should face the target to facilitate subsequent actions by the robotic arm.
To this end, the team proposed MAP (Manipulation Anchor Pose), explicitly defining the reference position and orientation suitable for subsequent manipulation as the learning objective for navigation.
Crucially, MAP supervision data can be generated at scale automatically.
The team leveraged HM3D, InteriorGS, and proprietary large-scale scene data, combining scene geometry information, vision-language models, and automated path planning to generate reference docking poses and approach trajectories for various manipulation targets.
Each trajectory includes not only visual observations and navigation actions but also geometric supervision signals such as target bounding boxes and MAP image projections.
The automatic generation pipeline for MAP-Data.
The resulting MAP-Data comprises over 1.5 million trajectories, totaling approximately 7,500 hours of data.
This means that robots do not need to rely entirely on real-world mobile manipulation demonstrations; instead, they can learn how to navigate to positions suitable for manipulation through large-scale synthetic data.
However, solving "where to stop" does not yet solve "how to learn together."
Mixed-Stream: Separate Learning Streams with Shared Model Capabilities
Navigation and manipulation require significantly different visual information, action spaces, and control mechanisms.
Simply concatenating the two types of actions may increase the difficulty of unified learning, while training two separate models makes it hard to fully share capabilities.
Fysiverse-UniWAM adopts an alternative approach: independent data streams sharing a core backbone network.
The overall architecture of Fysiverse-UniWAM's Mixed-Stream.
The navigation stream specifically processes environmental observations and chassis actions, while the manipulation stream handles multi-view observations and robotic arm actions.
Both data streams have their own input encoders and action output heads but share a unified Video DiT and Action DiT backbone. They achieve collaborative modeling of visual dynamics and action generation through a Mixture-of-Transformers (MoT) mechanism.
More importantly, navigation data and manipulation data do not need to come from the same complete mobile manipulation trajectory.
During training, the two types of data can be sampled independently and batched together to jointly update the shared backbone. During inference, both data streams can operate independently or execute in parallel.
This allows massive amounts of navigation data and fixed-station manipulation data to participate in unified model training, reducing reliance on complete mobile manipulation demonstration data.
Additionally, the model introduces auxiliary supervision on the image plane: the navigation stream predicts target bounding boxes and MAP projection locations, while the manipulation stream predicts the 2D projected trajectory of the robotic arm's end-effector. This further strengthens the connection between visual information and action prediction.
Visualization of navigation MAP prediction and manipulation end-effector trajectory prediction.
At this point, the technical logic of UniWAM becomes clear:
MAP solves where the robot should stop, while Mixed-Stream solves how navigation and manipulation can learn jointly.
By combining these two approaches, originally relatively independent tasks—navigation, mobile manipulation, and fixed-station manipulation—can be collaboratively modeled within a unified world-action model.
From benchmark tests to zero-shot transfer, how effective is this approach?
To validate this design, the team constructed MAP-Bench, comprising 15,000 evaluation trajectories, and compared it against models such as π₀.₅, GR00T N1.7, FastWAM, Uni-NaVid, and InternVLA-N1.
The results show that UniWAM-Scaled achieves the lowest docking error across short, medium, and long-distance tasks, with an overall average position error of just 9.85 cm and an orientation error of only 2.30°.
Compared to the strongest external baseline, the position error is reduced by 30.1%, and the orientation error is reduced by 44.0%.
Table 4 in the technical report presents a performance comparison of various models on MAP-Bench.
Moreover, this advantage does not stem entirely from additional data.
Even without incorporating MAP-Data, UniWAM-Base already outperforms external baselines; adding MAP-Data further improves docking accuracy.
Architectural ablation studies also demonstrate that Mixed-Stream improves mobile manipulation success rates and docking precision compared to alternatives such as unified action tokens, independent action experts, or sequence concatenation.
More interestingly, there is zero-shot real-world transfer.
The team trained UniWAM-Sim using only MAP-Data, without any real robot demonstration data, and then deployed it directly onto two different real robot platforms.
Ultimately, the model achieved a position error of 18.7 cm and an orientation error of 6.34°, demonstrating the potential for transferring capabilities from generative scene data to real robots.
Table 8 in the technical report displays the zero-shot real-world transfer results of UniWAM-Sim.
This further indicates that through explicit geometric supervision and unified model design, improvements in robotic capabilities do not rely solely on scaling up real-robot pretraining.
Unified Robotic Capabilities: Scaling Pretraining Is Not the Only Path
What makes Fysiverse-UniWAM truly noteworthy is not just its leading metrics in navigation and mobile manipulation.
More importantly, it demonstrates a distinct pathway for building robotic capabilities.
Through MAP, navigation is made explicitly task-oriented toward subsequent manipulation; through Mixed-Stream, disparate navigation and manipulation data are jointly learned within a single world-action model.
This does not imply that large-scale robot pretraining has become unimportant, but rather suggests that it may not be the sole prerequisite for constructing unified robotic capabilities.
From precise navigation to mobile grasping, and further to bimanual collaboration and fine-grained manipulation, Fysiverse-UniWAM uses 24 real-world tests to showcase the breadth of a unified model's capabilities, proving the potential of this technical approach.
As embodied intelligence moves beyond merely competing on the scale of robot data, how to truly integrate diverse capabilities may equally be key to achieving general-purpose robotics.
Finally, here are the project and open-source resources:
Project Homepage & Demos
https://fysics-ai.github.io/uniwam.github.io/
Technical Report
https://arxiv.org/abs/2609.39388
GitHub Code
https://github.com/Fysics-AI/UniWAM
Hugging Face Navigation Weights
https://huggingface.co/Fysics-AI/UniWAM
MAP-Data Dataset
https://huggingface.co/datasets/Fysics-AI/MAP-Data
MAP-Bench Data & Evaluation Trajectories
https://huggingface.co/datasets/Fysics-AI/MAP-Bench
© THE END
Please contact this official account for authorization to reprint.
Submissions or media inquiries: [email protected]