Apple's Robot Walked Into Rooms It Had Never Seen and Finished the Job

Apple's research group has published a paper describing HALO, a control system that lets a humanoid robot walk into a room it has never seen and do a job. The paper, “Perceptive Humanoid Motion from Simulation”, appeared on arXiv on 4 November, along with code for data generation, training and deployment.

The hardware is not Apple's. The team used a Unitree G1, the Chinese humanoid that has become a common research platform, and bolted a 3D-printed shell to its back containing a Mac mini with an M4 Pro chip and 64 GB of memory. The Mac runs the expensive parts: a vision-language model that decides what to do, and a diffusion model that generates the motion. The robot's own board handles the fast loop — a reinforcement-learning policy that tracks and executes the movement, 50 times a second.

The stack has three tiers. At the top, a vision-language model orchestrates the task, by default a Qwen model with 35 billion parameters, or a cloud model through an API. In the middle, a smaller Qwen model generates motion. At the bottom, the tracker turns that into joint commands.

The results are the interesting part. Across six real environments with no prior scanning or mapping, the system ran 360 single-task trials and succeeded on 91% of them on average. That is a high number for a robot working without a prepared map, and it is the kind of figure that only became reachable recently, as simulation started producing usable training data at scale.

Longer tasks were harder. In one real scene, three composite tasks were each run ten times, averaging 70% success. The hardest — duck under an obstacle, walk to a target, push a box — worked five times out of ten.

The training pipeline is costly but describable: 250 H100 GPUs for about 48 hours to synthesise the main dataset, 64 B200s for imitation learning on the motion generator, and 4 H100s for the reinforcement learning and distillation behind the tracker.

The paper is candid about what still needs people. New physical tasks still require a human to write prompts and design keyframes, and some motions came from human video or motion capture. More fundamentally, the robot relies on vision and cannot feel contact; when an object leaves the camera's view or is occluded, tasks that involve touching it get worse. Adding touch is the stated next step.