Huawei Noah Tops BEHAVIOR Challenge with 48.2% Task Success

How can classical robotics methods evolve into a scalable knowledge system?

Published by Machine Heart

Huawei Noah's Ark Embodied Intelligence Team's The North Star, with its HiPhys-RSI solution, stood out from 57 competing teams worldwide in the BEHAVIOR-1K Challenge 2026 international embodied intelligence competition, winning the championship by a decisive margin. In the hidden test covering 100 household tasks, it achieved a Q-score of approximately 0.683 and a full-task success rate of 48.2%.

This is the second edition of the challenge. Built on the BEHAVIOR-1K benchmark platform developed by Fei-Fei Li's team at Stanford University, it is held in conjunction with the top robotics conference CoRL 2026 (November 9–12). Centered on real household needs such as tidying and cleaning, it comprehensively evaluates robots' high-level reasoning, cross-room navigation, and dual-arm fine manipulation capabilities.

Fei-Fei Li herself also posted on social media, stating that robot learning is making remarkable progress and inviting more researchers to pay attention to the BEHAVIOR Challenge at CoRL 2026.

Compared with last year, the 2026 challenge achieved significant leaps in multiple dimensions: the benchmark scale doubled (tasks increased from 50 to 100, scenes from 3 to 7), and participation tripled (participants increased from 96 to 265, submissions from 18 to 57). Meanwhile, the best full-task success rate rose from 12% to 48%, four times that of the previous edition.

As a high-threshold "touchstone" for testing the comprehensive capabilities of embodied intelligence systems, BEHAVIOR-1K focuses on long-horizon tasks and ranks alongside Robodojo, which emphasizes scene generalization, as a T0-level embodied intelligence benchmark. Compared with short-duration grasping and single-skill evaluation, this challenge requires robots to rely on onboard vision, depth, and proprioceptive information to continuously complete multi-object interaction, state-change handling, and multi-skill coordination in minute-level long-horizon tasks, placing stringent demands on task planning, perception-action coordination, and execution stability.

12 successful task rollouts.

To tackle such complex challenges involving long-horizon, multi-skill coordination, the industry currently still mainly follows the Scaling path most familiar in AI, namely larger models, more data, and higher computational investment.

There is another path to capability growth in AI development: accumulating knowledge, rules, and operational procedures so that systems master methods that can be reasoned about, composed, and executed. Early symbolic AI explored this path for a long time, and the robotics field has also accumulated a wealth of perception, planning, and control tools.

Now that large models can already write programs, use tools, and modify plans based on execution results, has a new connection emerged between these two paths?

In the champion solution's technical report "Scaling by Structure: What Forms Can Scaling Take in Embodied AI?", the team proposes a broader Scaling question: can embodied intelligence continuously expand its capabilities by building and improving structured, executable knowledge and skills?

This work treats executable programs and the tools, constraints, and experience behind them as carriers of capability accumulation. Programming agents build, verify, and improve these structures; robots complete tasks through them; and the resulting trajectories in turn support vision-language-action model (VLA) training.

Letting "how many executable skills the system has mastered" also become a dimension of embodied intelligence capability growth.

How can classical robotics methods evolve into a scalable knowledge system?

The robotics field already possesses a large number of effective methods. Geometric reasoning handles spatial relationships and collisions, SLAM supports localization and mapping, task and motion planning connects logical decisions with action feasibility, and feedback control continuously corrects execution errors.

These methods each have their applicable conditions. When used for different tasks and environments, they require organizing information flow, connecting module interfaces, designing operation sequences, and adjusting strategies according to object geometry and scene constraints. After the scene changes, the original pipeline may require new grasping methods, action sequences, or recovery mechanisms.

This also constitutes the key bottleneck when structured robotic systems scale their capabilities: as tasks increase, the manual effort required to build and maintain solutions also grows.

Programming agents offer the possibility of automatically taking on part of this work. They can read execution logs, analyze videos, locate failures, modify tools and programs, and then run verification. Effective improvements are retained and become knowledge that subsequent tasks can invoke and adapt.

Thus, generalization takes on an additional meaning: the system can use existing tools and experience to build and adapt solutions for new tasks.

The scaling potential of this capability depends both on the number of tasks covered and on the engineering and interaction cost required for each additional task. The report examines both aspects through experiments.

HiPhys-RSI: Turning interaction experience into executable structures

The team proposes Hierarchical Physical Recursive Self-Improvement (HiPhys-RSI), a hierarchical physical recursive self-improvement framework. Built on a parameter-frozen coding agent and a vision-language model (VLM), it constructs and improves autonomously executed robotic pipelines from training instances.

Figure 1 | Overall framework of HiPhys-RSI. The agent accumulates executable knowledge through task experience, program construction, execution analysis, and iterative improvement; at inference time, it selects an existing pipeline or a VLA policy based on task conditions. (Report Figure 1)

First, the agent reads the task instructions and reference demonstrations, and gains interactive experience through tool calls: which operations work, what dependencies exist between steps, and what observations can indicate that the task is making progress.

These experiences are then organized into programs. How perception outputs are used for planning, when the controller executes, and how failures are handled are all connected through explicit information flow and conditional logic. The constructed pipeline can execute autonomously based on its own checks and branching logic.

Execution results then feed into the next round of analysis. Logs show what the program called, while videos reveal what actually happened in the physical world: after the trajectory completes, did the object really move into place? When a grasp command returns success, is the object still on the table? The agent combines this evidence to locate the modules and strategies that need modification.

Improvements can occur at the high level of tool selection, step orchestration, and recovery logic, or go deep into low-level tool implementations. The construction phase can also incorporate human corrections to provide guidance for complex problems. Validated programs and experiences are retained and can serve as reusable components for subsequent tasks.

The real iteration record for the same case, including failure phenomena, modifications made, and re-execution results; human involvement is noted.

Throughout this process, the parameters of the coding agent and VLM remain frozen, and improvements are stored in executable programs and structured knowledge. The final system uses capability cards to record the applicable conditions, historical performance, and known difficulties of each pipeline and VLA, and selects the appropriate execution components at inference time.

From tools, to processes, and then to processes that can be reused and composed, structured knowledge accumulates layer by layer.

100 household tasks, comparing two ways of growing capability

The team studies this path in the complex household simulation environment of BEHAVIOR-1K. The evaluation covers 100 tasks across eight capability categories, involving mobile pick-and-place, placement and sorting, fine manipulation, tool use, pouring, device operation, and object processing.

These tasks require continuous coordination of perception, decision-making, and action. Whether the agent can approach a target after finding it, whether it can carry an object after grasping it firmly, and whether it can proceed to the next step after completing one step all affect the final outcome.

Based on the 2025 BEHAVIOR Challenge winning solution, the team adapts and compares three VLA settings: joint training for one epoch on demonstration data from all 100 tasks (Meta-1-Epoch); per-task post-training starting from that checkpoint (Per-Task Post-Training); and joint training for two epochs on the same all-task demonstration data (Meta-2-Epochs). One epoch means a complete pass through the training data.

Per-task post-training mixes benchmark demonstrations with program-generated trajectories, trains an independent policy for each task, and introduces 3D inputs such as depth and geometric information relative to the robot's position. Therefore, its effects involve task adaptation, data supplementation, and representation changes simultaneously. The three VLA groups serve both as controls that execute tasks individually and as policies selectable by the final system at inference time.

Figure 2 | Comparison of the three VLA groups and HiPhys-RSI on 100 tasks. Each radial bar represents the score for one task; A–H correspond in order to mobile pick-and-place, general placement, matching placement, fine manipulation, tool use, pouring, device operation, and object processing. (Technical Report Figure 2)

In the experiments reported, the final hybrid HiPhys-RSI system achieves higher scores than the compared VLA settings on almost all tasks, with advantages across all eight task categories.

The different VLA settings reflect the role of training methods. Under the current configuration, an additional epoch of all-task joint training brings relatively limited gains; per-task post-training improves performance on some tasks, but the overall distribution of strengths and weaknesses across categories is largely preserved. These results reflect the performance of each configuration under the experimental conditions of this study.

The structured system, by contrast, handles these task requirements through explicit division of labor: perception and mapping modules provide scene information, programs organize task logic and constraints, and specialized planning and control methods complete actions. The coding agent adjusts their composition and implementation based on feedback, enabling existing robotic knowledge to adapt to more task conditions.

The experiments extend the discussion of Scaling to the entire system: how training increases policy capability, and how structure organizes and scales existing capabilities.

How structure scales: from capability composition to strategy discovery

Beyond the final score, a more critical question is: what does the growth process of structured knowledge look like?

Figure 3 | Pipeline improvement process. Left: overall task score changes with iterations; middle: number of iterations required for each task; right: frequency of use of different skills in the pipeline. (Report Figure 3)

The curves in the report show three stages: rapid improvement early on, then a plateau, and then some gains through continued exploration.

Early growth mainly comes from capability composition. Many problems can be solved by correcting tool configurations, execution order, and module coordination. Once existing capabilities are effectively organized, the system can cover a large number of tasks.

As these problems are gradually resolved, the remaining tasks require deeper strategy exploration. Continuing to tune the parameters of existing solutions may yield limited returns; new progress begins to depend on different contact methods, preparatory actions, and operation sequences. Some later gains come from discovering more stable manipulation methods that do not appear in the reference demonstrations.

Task 38, prioritizing a real comparison between the original strategy and the improved strategy, highlighting changes in contact methods, preparatory actions, and operation sequences.

The plateau may therefore signal a shift in the mode of improvement: from connecting existing capabilities together to exploring new solutions.

Structured Scaling thus involves both the reuse of general capabilities and the breakthrough of specific bottlenecks. The investment required to cover more difficult tasks may grow faster than the number of tasks, making how to allocate the improvement budget an important question.

Can structure transfer, and what further limits improvement?

The hidden evaluation tests whether programs can transfer to new instances beyond development. Q-scores decreased on most tasks, with most drops below 0.2; some tasks maintained similar performance or achieved higher scores, demonstrating a degree of cross-instance transfer capability. What is examined here is the performance of known tasks under different initial conditions.

Figure 4 | Difference between hidden evaluation and previously reported Q-scores (Qhidden − Qreported). Each bar corresponds to one task; positive values indicate higher hidden evaluation scores, negative values indicate decreases. (Technical report Figure 5)

Larger drops occurred mainly in tasks that rely on object search. Changes in initial object positions, occlusion, and hard-to-reach areas can all significantly increase search and manipulation costs. Some tasks have few scoring conditions or even use binary scoring, so a single unmet condition can cause a large score change; with only 10 test instances per task, there is also some sampling fluctuation.

Failure modes reveal the bottlenecks for further improvement. In the failure type ranking for the VLA baseline, grasping and placing rank at the top; in HiPhys-RSI, grasping and timeout rank at the top, while placing, articulated object manipulation, and object dropping rank relatively lower.

Figure 5 | Comparison of failure modes. Numbers inside the bars indicate the frequency ranking of that mode across methods. Grasping remains a common difficulty, and timeout and navigation issues are also prominent in HiPhys-RSI. Categories are not mutually exclusive; multiple failure modes may occur in a single execution. (Technical report Figure 6)

When the target is hidden in an invisible location, or when the passage and manipulation space are too narrow, repeated search and planning continuously consume the execution budget. Code improvements remain constrained by current perception capabilities, search strategies, physical reachability, and execution time. This indicates that the accumulation of structure can solve many coordination and execution problems, while continued growth also requires further improvement of these underlying capabilities and ways of interacting with the environment.

How do new manipulation strategies become system knowledge?

Specific cases demonstrate the content of structured knowledge. Through agent-proposed solutions and human-guided improvement, the system developed manipulation approaches adapted to different tasks.

Figure 6 | Manipulation strategies formed during program construction and improvement. Top row: geometric adjustment of wooden supports, spray bottle preprocessing; middle row: grasping the pot rim, scanner operation and movement; bottom row: propping up a paper bag with a box, carrying a wheelbarrow from the front edge of the bed. The wood task is also used in the evaluation loophole analysis later. (Technical report Figure 7)

When the pot handle is hard to reach, the robot grasps the near-side pot rim, keeps the pot level, and then carries it onto the stove. When the wheelbarrow handle is blocked, the robot approaches from the wheel side and grasps the front bed edge with both hands to complete the transport.

When the spray bottle is upright and difficult to grasp and activate, the system first changes its orientation and then operates it from a more suitable direction. When packing takeout, the robot first stands the empty paper bag inside the box, then places the burger in it, using environmental support to reduce manipulation difficulty. For the scanner, it can first be activated on the desktop, and then moved by pushing or grasping depending on positional conditions.

Task 81 & Task 92, demonstrating preparatory actions, operation sequences, and final results.

These experiences include contact point selection, object state adjustment, use of environmental support, and action sequence arrangement. Once written into programs, they can be reused during autonomous execution.

Broader strategies also expose gaps in task scoring. In the wood stacking task, the system arranged the wood into separate pairs, satisfying the contact condition but failing to form the expected wood pile—a fairly typical exploitation of an evaluation loophole. Another case is cutting food directly inside the refrigerator: it obtained partial scores but did not complete the subsequent transfer goal, and its behavior could be either a valid intermediate step or a stage of stopping at easy scores.

Figure 7 | Evaluation issues exposed by strategy exploration. Left: using separate wood pairs to satisfy the contact condition; right: cutting food inside the refrigerator obtains partial scores but does not complete the subsequent goal. (Technical report Figure 8)

These observations also prompt reflection on evaluation: success conditions need to accurately express task intent while preserving room for legitimate alternative solutions. Whether the system can judge stage progress and task completion on its own, as well as the accuracy and latency of such judgment, also deserves evaluation as an independent capability.

Connecting structured experience with model training

The experience generated by program improvement can also enter model training through execution trajectories.

The team mixed observation–action sequences generated by the pipeline with benchmark demonstrations for per-task VLA post-training. The manipulation approaches and coordination experience formed during program exploration thus become data that learning policies can leverage; the trained policies, together with the existing pipeline, participate in inference-time selection.

This provides a foundation for further co-evolution of structure and policy: stronger policies can expand the capabilities available for subsequent system construction, and continuously improving pipelines can continue to generate training experience. This bidirectional process remains a direction worth exploring in depth.

This technical report further asks: in how many forms can the capability growth of embodied intelligence occur?

Model training lets experience enter parameters, while program construction turns experience into executable knowledge. Coding agents connect the two, enabling the tools and methods accumulated in the robotics field to serve a broader range of tasks through automated construction, verification, and reuse.

Scaling by Structure explores exactly this dimension: expanding the capabilities of the entire embodied system by continuously accumulating and improving structured, executable knowledge.

Core Related Technologies

[1] Jinbang Huang, Zhiyuan Li, Yuanzhao Hu, Ran Qi, Yixin Xiao, Mark Coates, Tongtong Cao, Zhanguang Zhang, and Yingxue Zhang. RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning. arXiv.18060, 2026. ECCV 2026 Agent in the World Workshop Best Paper Award.

[2] Ran Qi, Yixin Xiao, Dawei Wang, Zhiyuan Li, Jinbang Huang, Xuan Zhao, Yintao Ma, Yangzheng Wu, Tongtong Cao, and Yingxue Zhang. RoboCoEvo: Expanding Robot Capabilities through Agent–Policy Co-Evolution. 2026. Preprint.

[3] Liheng Ma, Rui Heng Yang, Amin Abyaneh, George Z. Xue, Behnam Rahmati, Mateo Clemente, Ziwen Hu, Anlin Chen, Tongtong Cao, Zhanguang Zhang, and Yingxue Zhang. Faster-WAM: Do World Action Models Need Deep Action Modules? arXiv.02365, 2026.

[4] Jinbang Huang, Zhiyuan Li, Yuanzhao Hu, Zhanguang Zhang, Mark Coates, Xingyue Quan, and Yingxue Zhang. Self-CriTeach: LLM Self-Teaching and Self-Critiquing for Improving Robotic Planning via Automated Domain Generation. International Conference on Machine Learning (ICML), 2026.

[5] Jinbang Huang, Yuanzhao Hu, Zhiyuan Li, Ran Qi, Yixin Xiao, Yangzheng Wu, Tengyue Ba, Zhanguang Zhang, and Yingxue Zhang. StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic Distillation. arXiv.20791, 2026.

Related Links

• Technical Report: https://behavior1k-champion.com/assets/HiPhys-RSI_tech_report.pdf
• Official Leaderboard: https://huggingface.co/spaces/behavior-1k/2026-challenge-leaderboard
• Evaluation Metrics and Rules: https://behavior.stanford.edu/challenge/evaluation.html

© THE END

For reprints, please contact this official account for authorization.

Submissions or press inquiries: [email protected]

Open in WeChat