RynnUnify: Towards Unified Embodied Brain
Abstract
We introduce RynnUnify, a 16B unified embodied model that combines strong embodied understanding with long-horizon visual planning through interleaved video--language reasoning. With capabilities spanning spatial reasoning, physical prediction, and grounding, RynnUnify connects an understanding of the physical world with visual anticipation of how it evolves across successive subtasks. Building on these capabilities, we further develop an agentic extension through reinforcement learning to selectively use video predictions for reasoning about unobserved states and interaction outcomes. We also develop a world-action extension of the full RynnUnify model, translating its understanding and prediction capabilities into robot control. To evaluate these capabilities, we introduce Rynn-VP and Rynn-GEU for long-horizon visual planning and anticipatory reasoning. Experiments show that RynnUnify achieves the highest average embodied understanding score among the evaluated unified multimodal models and improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano. Its agentic extension improves the Rynn-GEU average by 6.0 points over direct reasoning, while the world-action extension achieves 51.5\% success on 20 selected composite RoboCasa365 tasks and 46.7\% average success across three real-robot tasks. Together, these results establish RynnUnify as a shared foundation for embodied understanding, visual planning, and robot control, taking a step toward a unified embodied brain.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
RynnUnify introduces a 16B unified embodied model that integrates spatial reasoning, physical prediction, and long-horizon visual planning, achieving state-of-the-art results on new benchmarks and demonstrating viable robot control capabilities. The paper presents a comprehensive framework for embodied AI, combining strong multimodal understanding with agentic reasoning and action generation, marking a significant step toward a unified embodied brain, although real-world robot success rates remain moderate.
Methodology
The paper proposes RynnUnify, a 16B parameter unified embodied model. The core methodological contribution is the integration of embodied understanding (spatial reasoning, physical prediction) with long-horizon visual planning via interleaved video-language reasoning. The authors introduce an agentic extension using reinforcement learning to selectively utilize video predictions for reasoning about unobserved states. Furthermore, a "world-action" extension is developed to translate these capabilities into robot control. The architecture appears to be a large multimodal model (likely VLM-based) fine-tuned on specific embodied datasets. The novelty lies in the unification of these distinct tasks (understanding, planning, control) within a single model framework, rather than a novel fundamental architecture.
Results & evaluation
The evaluation is extensive, introducing two new benchmarks: Rynn-VP for long-horizon visual planning and Rynn-GEU for anticipatory reasoning. The model achieves the highest average embodied understanding score among evaluated unified multimodal models. It improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano. The agentic extension improves Rynn-GEU by 6.0 points. In robotics, the world-action extension achieves 51.5% success on 20 composite RoboCasa365 tasks and 46.7% average success on three real-robot tasks. These results demonstrate significant progress in unifying perception and action, though the success rates in real-world robotics remain moderate, indicating room for improvement.
Reproducibility
The paper is an arXiv preprint. While the abstract mentions specific models and benchmarks, the provided text does not include explicit links to code or data repositories. Without access to the specific training data mixes, hyperparameters, and the implementation of the RL agentic extension, full reproducibility is difficult to guarantee. The scale of the model (16B) and the complexity of the embodied tasks make independent reproduction resource-intensive.
Limitations
The real-robot success rates (46.7% average) suggest that while the model is a strong foundation, it is not yet robust enough for fully autonomous deployment in complex, unstructured environments. The reliance on specific benchmarks (Rynn-VP, Rynn-GEU) limits immediate generalizability to other evaluation suites. The "unified" nature may come at the cost of specialization compared to dedicated models for specific sub-tasks.
Broader impact
This work contributes to the trend of general-purpose embodied agents. By unifying understanding and planning, it provides a scalable framework for developing robots that can reason about physical interactions. The introduction of new benchmarks for visual planning and anticipatory reasoning will likely be adopted by the community to evaluate future embodied models. RynnUnify introduces a 16B unified embodied model that integrates spatial reasoning, physical prediction, and long-horizon visual planning, achieving state-of-the-art results on new benchmarks and demonstrating viable robot control capabilities. The paper presents a comprehensive framework for embodied AI, combining strong multimodal understanding with agentic reasoning and action generation, marking a significant step toward a unified embodied brain, although real-world robot success rates remain moderate.
Author of this paper? Send a correction.