Apple Unveils HALO Humanoid Robot System With 91% Success Rate in Unmapped Environments
Apple's research team published an arXiv paper detailing HALO, a humanoid robot autonomy stack that layers a VLM task orchestrator, a DiT motion generator, and a reinforcement-learning tracker, deployed on a Unitree G1 with an onboard Mac mini for inference. Across 360 single-task trials in six previously unmapped real environments, HALO averaged a 91% success rate, though long-horizon composite tasks fell to 70% and the hardest combination to 50%. The team also open-sourced the data generation, training, and deployment code.
Full Report
IT之家 November 7 news: On November 4 local time, Apple's research team submitted the paper "Perceptive Humanoid Motion from Simulation" to the preprint platform arXiv, disclosing the humanoid robot autonomous control system HALO.
The research team also released the data generation, model training, and deployment code, covering motion generation, motion tracking, task orchestration, and simulation and real robot operation, among other aspects.
In the deployment phase, the research team used the Unitree G1 humanoid robot, with a dedicated 3D-printed protective shell mounted on its back, housing a Mac mini (M4 Pro chip, 64GB unified memory). The Mac mini is responsible for real-time on-board inference of the computationally expensive Orchestrator (VLM) and Motion Generator (DiT); the G1's built-in onboard chip focuses on running the high-frequency Tracker (RL policy network).
When executing tasks, HALO is controlled collaboratively by three levels. The top layer is the task orchestration module based on a vision-language model (VLM), which by default uses Qwen3.6-35B-A3B; the system can also optionally call cloud models via API. The middle motion generation layer uses another Qwen model, Qwen3.5-2B. The bottom layer is the motion tracker trained through reinforcement learning, policy distillation, and other steps, running at a frequency of 50 times per second.
According to the paper, the research team conducted 360 single-task tests in 6 real environments that were not pre-scanned or mapped, achieving an average success rate of 91%.
For long-sequence composite tasks, in a single real scenario, 10 tests were conducted for each, and the average success rate across 3 combinations was 70%; among them, the most complex one—"bend down to pass through an obstacle — walk toward the target — push the box"—succeeded 5 times out of 10 tests, with a success rate of 50%.
According to the introduction, the research team used 250 NVIDIA H100 GPUs and took about 48 hours to synthesize the main training data. The motion generator was trained via imitation learning using 64 B200s; the motion tracker was trained through reinforcement learning, policy distillation, and other steps using 4 H100s.
However, simulation-generated data has not completely eliminated human involvement. The team stated that adding new physical tasks still requires manually writing prompts and designing motion keyframes, and some motions also used human videos or motion capture footage.
The research team also pointed out that the robot currently relies mainly on visual perception and has difficulty directly perceiving contact conditions; when objects leave the camera's field of view or are occluded, performance on tasks involving more contact degrades. Adding tactile perception will be one of the directions for future improvement.
IT之家 attaches the relevant links:
- Paper: https://arxiv.org/abs/2611.06908
- GitHub repository: https://github.com/apple-aiml-research/ml-halo
Advertising statement: External redirect links contained in the article (including but not limited to hyperlinks, QR codes, passwords, and other forms) are used to convey more information and save selection time; the results are for reference only. All articles on IT之家 containing external links include this statement.