Heave-Ho: Decentralized Long-Horizon Multi-Humanoid Collaboration
Abstract
Collaboration promises to extend humanoid capabilities beyond what individual robots can achieve. However, realizing this potential requires coordinated whole-body interactions, coherent long-horizon planning, and precise synchronization across robots. We introduce Heave-Ho, a decentralized framework for long-horizon collaborative multi-humanoid loco-manipulation. At the low level, we develop a multi-agent post-training framework that adapts a humanoid foundation tracking model to physically coupled interactions between humanoids, enabling coordinated decentralized control from local observations alone. At the high level, we train a vision-language planner and design a speech-based leader-follower coordination mechanism to synchronize decentralized execution without network communication. Trained with two identical humanoids, our system scales to teams of up to ten robots and generalizes to heterogeneous teams without retraining. To benchmark these capabilities, we introduce HH-Bench, comprising 12 tasks involving teams of 2 to 10 humanoids, including heterogeneous teams. Systematic evaluations show that Heave-Ho consistently outperforms baseline methods, while real-world experiments demonstrate robust decentralized collaboration across challenging tasks.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces Heave-Ho, a decentralized framework for long-horizon multi-humanoid collaboration that combines a multi-agent post-trained low-level controller with a vision-language high-level planner and a novel speech-based synchronization mechanism, demonstrating scalable and robust performance across heterogeneous teams of up to ten robots.
Methodology
The paper proposes a two-tiered decentralized framework for multi-humanoid collaboration. The low-level component utilizes a multi-agent post-training strategy to adapt a pre-trained humanoid foundation model for physically coupled interactions, allowing robots to coordinate via local observations without explicit communication. The high-level component employs a vision-language model (VLM) as a planner that generates discrete team skills, which are then synchronized across the team using a speech-based leader-follower mechanism. This speech-based synchronization is a creative, albeit unconventional, solution to the latency and bandwidth constraints of networked multi-robot systems, effectively using the acoustic channel as a coordination medium. The approach is novel in its integration of VLM planning with decentralized physical control for long-horizon tasks.
Results & evaluation
The authors introduce HH-Bench, a benchmark comprising 12 tasks involving teams of 2 to 10 humanoids, including heterogeneous teams. The evaluation demonstrates that the system scales from 2 to 10 robots and generalizes to heterogeneous teams without retraining, which is a significant claim. Real-world experiments on PM01 humanoids show robust performance. However, the reliance on speech for synchronization introduces environmental dependencies (noise, distance) that are not fully quantified in the provided abstract/summary. The comparison to baselines is claimed to be consistent, but specific quantitative metrics (success rates, time-to-completion) are not detailed in the provided text, making it hard to assess the magnitude of improvement.
Reproducibility
The paper mentions a project page with videos, which aids in qualitative verification. However, the specific details of the "humanoid foundation tracking model" (whether it is open-source or proprietary) and the exact implementation of the speech-based coordination protocol are crucial for reproducibility. The use of specific hardware (PM01) limits immediate reproducibility for labs without access to such platforms. The code availability is not explicitly stated in the provided text, though a project page exists.
Limitations
The primary limitation is the reliance on speech for synchronization, which is susceptible to environmental noise and may not scale well in very large or noisy environments. Additionally, the system is tested on identical humanoids (PM01) for training, and while it claims generalization to heterogeneous teams, the extent of this generalization (e.g., different kinematics, different control policies) is a critical area of uncertainty. The long-horizon nature of the tasks is a strength, but the failure modes of the VLM planner in complex, dynamic environments are not deeply analyzed.
Broader impact
This work has significant implications for the deployment of humanoid robots in collaborative settings, such as logistics, construction, or disaster response, where centralized control is impractical. The decentralized approach reduces infrastructure requirements (no need for high-bandwidth networks between robots). The integration of VLMs for high-level planning in physical robotics is a trending and impactful direction. The introduction of HH-Bench provides a useful resource for the community to evaluate multi-robot collaboration. The paper introduces Heave-Ho, a decentralized framework for long-horizon multi-humanoid collaboration that combines a multi-agent post-trained low-level controller with a vision-language high-level planner and a novel speech-based synchronization mechanism, demonstrating scalable and robust performance across heterogeneous teams of up to ten robots.
Author of this paper? Send a correction.