Machine Learning PapersLandmark papers
← All papers
RESEARCH PAPER / 2026-11-13

Training Reasoning Models in Simulation to Natively Use Real Robots

Jiaming Tang, Zhuoyang Zhang, Junxian Guo, Fangzhou Zhao, Yilong Zhao, Zekai Li, Xingyang Li, Yuyang Zhao, Song Bian, Zhijian Liu, Song Han

MIT +4 · ICML 2026

Affiliations: MIT, UC Berkeley, UCSD, NVIDIA, Inco AI

Abstract

Robot-use agents (RUAs) have emerged as a promising approach to general-purpose robot control. However, strong performance has so far been achieved only by frontier proprietary models such as GPT-6 Astra and Claude Fable 5.2, whereas open-weight models remain substantially less capable. We show that an open-weight 27B reasoning model can be trained to use real robots natively through reinforcement learning, entirely in simulation and without demonstrations. Like a computer-use agent (CUA), the model observes the environment, reasons in a chain of thought, and generates the next end-effector pose, without any harness. We build a Post-training Engine for Agentic Robotics (PEAR) and use it to train the model on simulated robot tasks at scale with robot-use reinforcement learning (RURL), mirroring how frontier models acquire computer use. We find that the skills acquired in simulation generalize well beyond the training tasks: they transfer to an unseen robot, to the real world, and to novel instructions. After training on LIBERO and RoboDojo in simulation, the model succeeds in 60% of our trials on a real dual-arm robot, whereas the base model succeeds in none. On LIBERO, training raises the success rate from 1.6% to 74.4%, approaching GPT-6 Astra and GPT-6.1 Sol at maximum reasoning effort. Compared with $π_{0.5}$, a vision-language-action model fine-tuned on demonstrations, our model is substantially more robust to changes in the task itself. On LIBERO-Pro, $π_{0.5}$ succeeds in only 0.75% of episodes when assigned a new task in a familiar scene whereas our model succeeds in 57.4%. Finally, the model retains its general capabilities, with no measurable degradation on nine reasoning, coding and vision benchmarks. These results highlight robot-use reinforcement learning as a promising component of foundation-model post-training.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

The paper presents a compelling and rigorous demonstration that open-weight reasoning models can be trained to natively control real robots via reinforcement learning in simulation, achieving performance competitive with proprietary frontier models. The combination of high novelty in applying RURL to large reasoning models and strong empirical results in sim-to-real transfer justifies a high impact score, positioning this as a key contribution to the field of agentic robotics.

Methodology

The paper introduces PEAR (Post-training Engine for Agentic Robotics), a framework for training open-weight reasoning models to control robots via reinforcement learning in simulation. The core methodological contribution is the application of Robot-Use Reinforcement Learning (RURL) to a 27B parameter model, enabling it to generate end-effector poses directly from observations and chain-of-thought reasoning without external harnesses. This approach mirrors the "computer-use" paradigm but applies it to physical robotics, leveraging the model's native reasoning capabilities to plan actions. The method relies on scaling RL in simulation (LIBERO, RoboDojo) to induce generalizable skills that transfer to real-world hardware.

Results & evaluation

The experimental results are striking. The model achieves a 74.4% success rate on LIBERO, up from 1.6% for the base model, approaching the performance of proprietary frontier models (GPT-6 Astra). More impressively, it demonstrates strong sim-to-real transfer, achieving 60% success on a real dual-arm robot where the base model had 0% success. The paper also highlights robustness to task variations, significantly outperforming the vision-language-action model $\pi_{0.5}$ on LIBERO-Pro (57.4% vs 0.75%). The evaluation includes checks for capability retention on non-robotic benchmarks, showing no degradation.

Reproducibility

The authors provide a GitHub repository link (https://github.com/mit-han-lab/pear), which is a strong indicator of reproducibility. However, training a 27B model with RL in simulation requires significant computational resources that may be prohibitive for many labs. The specific hyperparameters and simulation environments are likely detailed in the code, but the barrier to entry for full reproduction remains high due to compute costs.

Limitations

The primary limitation is the computational cost of training large reasoning models with RL. Additionally, while the model generalizes to unseen robots and tasks, the scope of "general-purpose" control is still limited to the specific types of manipulation tasks tested. The reliance on simulation for training raises questions about the coverage of the simulation environment relative to the full complexity of real-world physics and sensor noise, although the 60% real-world success rate mitigates this concern to some extent.

Broader impact

This work is highly significant as it bridges the gap between open-weight LLMs and physical robotics. By demonstrating that open models can be post-trained to use robots natively via RL, it challenges the dominance of proprietary models in this domain. It provides a scalable pathway for the community to develop capable robot agents without relying on massive demonstration datasets, potentially accelerating research in general-purpose robot control. The paper presents a compelling and rigorous demonstration that open-weight reasoning models can be trained to natively control real robots via reinforcement learning in simulation, achieving performance competitive with proprietary frontier models. The combination of high novelty in applying RURL to large reasoning models and strong empirical results in sim-to-real transfer justifies a high impact score, positioning this as a key contribution to the field of agentic robotics.

Author of this paper? Send a correction.