Machine Learning PapersLandmark papers
← All papers
RESEARCH PAPER / 2026-11-14

RoboCoEvo: Expanding Robot Capabilities through Agent-Policy Co-Evolution

Ran Qi, Yixin Xiao, Dawei Wang, Zhiyuan Li, Jinbang Huang, Xuan Zhao, Yintao Ma, Yangzheng Wu, Tongtong Cao, Yingxue Zhang

Huawei Noah's Ark Lab +1 · arXiv

Affiliations: Huawei Noah's Ark Lab, University of Toronto

Abstract

Integrating MLLM-based agents with robotic foundation models has attracted increasing attention as a means of enhancing robotic capabilities. However, the improvement of such integrated systems remains largely decoupled across their agent and model components, with advances in one component not necessarily benefiting the other, substantially limiting the capabilities of the combined system. As both MLLMs and robotic foundation models become increasingly capable, fully realizing their combined potential calls for co-evolution, an iterative process in which improvements in agents and robotic foundation models reinforce one another. We propose RoboCoEvo, a failure-guided co-evolution framework that couples the refinement of the agent's explicit and reusable executable knowledge with parameter learning in robotic foundation models, such as vision-language-action (VLA) models. Through a code-as-policy formulation, the agent develops and refines executable failure-detection and recovery programs, generating targeted corrective experience. Post-training consolidates the resulting corrective behaviors into the robotic model's end-to-end policy, improving autonomous task execution. Changes in the updated policy's failure distribution, in turn, guide subsequent revisions to the agent's programs. Experiments on BEHAVIOR-1K, RoboCasa, and RoboTwin demonstrate sustained improvements in autonomous VLA task performance across successive co-evolution rounds.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

RoboCoEvo introduces a failure-guided co-evolution framework that couples MLLM agent refinement with VLA model post-training to enhance robotic capabilities. The paper demonstrates significant improvements in task success rates across multiple benchmarks by iteratively generating targeted corrective data from failure modes, highlighting the potential of agent-policy co-evolution for advancing robotic foundation models.

Methodology

The paper proposes RoboCoEvo, a framework for co-evolving MLLM-based agents and Vision-Language-Action (VLA) models. The core method involves a "code-as-policy" formulation where the agent generates executable failure-detection and recovery programs. These programs are used to generate targeted corrective data from failed rollouts, which is then used to fine-tune the VLA model. The process is iterative: the updated policy's new failure distribution guides the agent to refine the programs in subsequent rounds. The methodology is sound, leveraging the reasoning capabilities of LLMs to handle high-dimensional state spaces where traditional RL might struggle with credit assignment or exploration. The use of shared failure-mode programs rather than per-failure programs is a key technical insight, reducing data entropy and improving learning stability.

Results & evaluation

The experiments are conducted on three major benchmarks: BEHAVIOR-1K, RoboTwin 2.0, and RoboCasa. The baselines are strong, including the first-place solution from the 2025 BEHAVIOR Challenge (RLC) and state-of-the-art VLAs like Pi0.5 and XR-1. The results show consistent improvements in success rates and Q-scores across multiple co-evolution rounds. Ablation studies effectively demonstrate the importance of targeted data versus untargeted data, the benefit of shared failure-mode programs over per-failure programs, and the efficiency gains from failure-conditioned augmentation. The failure-mode analysis provides detailed insights into how specific failure rates decrease and how the policy learns to recover from them.

Reproducibility

The paper provides detailed hyperparameters for training on each benchmark, including learning rates, batch sizes, and optimizer settings. It also details the data generation process, including the number of rollouts, the agent model used (GPT-6 Astra), and the cost accounting for GPU hours and agent tokens. The code-as-policy approach is described in detail, though the specific prompts and code structures are likely in the appendix or code repository (not provided in text). The reproducibility is high given the detailed setup, assuming access to the specific agent model and simulation environments.

Limitations

The method relies on a powerful external agent (GPT-6 Astra) for code generation, which may be costly and not always available. The framework is currently limited to simulation environments; transferring to real-world robots would require additional work. The paper acknowledges that it only learns from successful recovery demonstrations and does not yet leverage reinforcement learning for credit assignment on failed rollouts. The iterative nature of the process may be time-consuming for complex tasks.

Broader impact

This work bridges the gap between high-level reasoning (LLMs) and low-level control (VLAs), offering a scalable path to improve robotic capabilities without massive amounts of human-labeled data. The co-evolution paradigm could be applied to other domains where agents and models can mutually improve. The focus on failure-driven learning is particularly relevant for safety-critical robotic applications. RoboCoEvo introduces a failure-guided co-evolution framework that couples MLLM agent refinement with VLA model post-training to enhance robotic capabilities. The paper demonstrates significant improvements in task success rates across multiple benchmarks by iteratively generating targeted corrective data from failure modes, highlighting the potential of agent-policy co-evolution for advancing robotic foundation models.

Author of this paper? Send a correction.