Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents
Abstract
Recent studies of embodied agents based on frontier multimodal large language models (MLLMs) develop harness engineering around the MLLM, such as harnesses with post-trained policies and recursive self-improvement loops. Yet whether these additions exceed the native capability of the MLLM remains unexplored. We introduce SimplEvo to explore how far an MLLM can improve under direct control when it retains its execution experience in self-written text memories. It requires no weight updates, post-trained policies, or privileged information. The MLLM writes a coordinate-free task memory once from a single demonstration. Under a strict train-test protocol, it then creates a bounded working memory on training episodes only and refines it after each episode with lessons on procedure, completion, and recovery. Both memories stay fixed at test time. On EBench, SimplEvo lifts the success rate (SR) of GPT-6-Astra under direct control from 46.7% to 81.7%, and tops the RoboDojo leaderboard with 69.63% SR and a score of 75.32. Compared with memories that also store past visual frames, SimplEvo reaches a similar SR with text memory alone while using fewer tokens. With its memories fixed, SimplEvo also generalizes to unseen layouts and randomized scene configurations. Moreover, the working memory transfers across models: written by GPT-6-Astra, it can be reused as is by weaker MLLMs and improve their performance without any adaptation of their own.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
SimplEvo demonstrates that a frozen MLLM can significantly improve embodied task performance through self-written, refined text memories without weight updates. The paper presents a compelling case for "harness engineering" over traditional policy learning, showing strong results on EBench and RoboDojo, along with notable cross-model memory transferability, though its reliance on specific frontier models and limited open-source details temper its immediate impact.
Methodology
The paper proposes "SimplEvo," a framework for embodied agents that relies on "harness engineering" around a frozen Multimodal Large Language Model (MLLM) rather than weight updates. The core method involves two types of text-based memories: a "task memory" written once from a single demonstration (coordinate-free) and a "working memory" that is refined after each training episode using lessons on procedure, completion, and recovery. The approach is novel in its strict separation of training (memory refinement) and testing (fixed memories), and in its claim that text-only memory is sufficient to match or exceed performance of methods using visual frame storage, while using fewer tokens. The methodology is interesting as it challenges the necessity of post-trained policies or privileged information, focusing instead on the MLLM's native capability to self-improve via textual reflection.
Results & evaluation
The experiments report significant improvements on EBench, lifting GPT-6-Astra's success rate from 46.7% to 81.7%. It also claims to top the RoboDojo leaderboard with 69.63% SR. The paper demonstrates generalization to unseen layouts and randomized scenes, and notably shows that the working memory written by a strong model (GPT-6-Astra) can be transferred to weaker MLLMs without adaptation, improving their performance. This cross-model transferability is a strong empirical finding. However, the reliance on specific, potentially proprietary or future models (GPT-6-Astra) and the lack of open-source baselines for direct comparison in the provided text limits the immediate verifiability of the results.
Reproducibility
The paper describes the protocol (single demonstration, strict train-test split, fixed memories at test time) clearly. However, the specific implementation details of how the MLLM is prompted to write and refine the memories, the exact format of the "lessons," and the specific hyperparameters for the memory refinement loop are not fully detailed in the abstract or provided text. The use of "GPT-6-Astra" suggests a specific, possibly non-public or very recent model, which may hinder reproducibility for the broader community unless the code and prompts are released.
Limitations
The primary limitation is the dependence on a specific, high-capability MLLM (GPT-6-Astra) for the results. It is unclear how well this method scales to smaller, open-source MLLMs without the "transfer" benefit, or if the transfer benefit diminishes with larger capability gaps. Additionally, the method is restricted to tasks where a single demonstration is sufficient to initialize the task memory, which may not hold for highly complex or multi-stage tasks. The "coordinate-free" nature of the memory is a strength but also a limitation if precise spatial reasoning is required beyond what the text can capture.
Broader impact
This work has significant implications for the design of embodied AI systems, suggesting that complex post-training or policy learning may be unnecessary if the LLM is given a structured framework for self-reflection and memory management. It supports the trend towards "harness engineering" and could lead to more modular, interpretable, and adaptable robotic agents. The finding that text memory can replace visual frame storage for similar performance has implications for computational efficiency and token usage in long-horizon tasks. SimplEvo demonstrates that a frozen MLLM can significantly improve embodied task performance through self-written, refined text memories without weight updates. The paper presents a compelling case for "harness engineering" over traditional policy learning, showing strong results on EBench and RoboDojo, along with notable cross-model memory transferability, though its reliance on specific frontier models and limited open-source details temper its immediate impact.
Author of this paper? Send a correction.