ReTerm: Recovery Training for Self-Improving Terminal Agents
Abstract
Terminal agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long-running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self-improving terminal agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm-10K, a high-quality dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm-GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self-recovery attempts. By learning from whether these attempts successfully complete the task, the agent self-improves through its own recovery experience. Built on Qwen3.5-9B, our ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
Methodology
The paper proposes ReTerm, a framework for training terminal agents to recover from their own execution errors. The methodology consists of two main components: (1) ReTerm-10K, a dataset constructed by collecting failed trajectories from a base model (Qwen3.5-9B), restoring the environment state, and using a stronger teacher model (DeepSeek-V4-Pro) to generate recovery continuations, which are then filtered for quality; and (2) ReTerm-GRPO, an online reinforcement learning algorithm that forks from the agent's own failure states during training to generate multiple independent self-recovery attempts. The use of a "recovery task queue" to manage the distribution of failure states across training steps is a practical engineering contribution that addresses the sparsity of failure events in RL. The approach is logically sound, addressing the specific gap of "recovery" rather than just "task completion," and the distinction between SFT cold-start and online RL self-improvement is well-motivated.
Results & evaluation
The experiments are comprehensive, evaluating the model on Terminal-Bench 2.0, OpenThoughts-TBLite, SWE-bench Verified, and out-of-domain tasks like AIME and SpreadsheetBench. The results show significant improvements over the base model (Qwen3.5-9B) and standard GRPO baselines. The ablation study clearly isolates the contribution of the recovery-specific RL component. The error analysis section, which categorizes errors into planning, tool use, implementation, verification, and resource management, provides valuable insight into where the model improves. The case studies are detailed and illustrative of the recovery process. The performance gains are substantial (e.g., +14.6 points on Terminal-Bench 2.0), suggesting a meaningful impact on the state of the art for 9B-scale models.
Reproducibility
The authors release code, data, and models, which is a strong positive for reproducibility. The paper provides detailed descriptions of the data pipeline, filtering criteria, and training hyperparameters. The use of specific model versions (Qwen3.5-9B, DeepSeek-V4-Pro) and benchmarks (Terminal-Bench 2.0) allows for clear replication. The prompt templates for data curation and error analysis are included in the appendix, further enhancing transparency.
Limitations
The method relies on a stronger teacher model (DeepSeek-V4-Pro) for the SFT cold-start phase, which may limit scalability if such a teacher is not available. The recovery task queue mechanism adds complexity to the training loop. The evaluation is primarily on terminal/coding tasks; while out-of-domain generalization is tested, the core contribution is specific to interactive terminal environments. The paper does not extensively discuss the computational cost of maintaining and sampling from the recovery task queue compared to standard RL.
Broader impact
This work has significant implications for the development of autonomous coding agents. By focusing on recovery, it addresses a critical failure mode in long-horizon tasks. The techniques could be extended to other interactive domains where agents must correct their own mistakes, such as robotics or GUI agents. The release of the ReTerm-10K dataset and the ReTerm-GRPO algorithm provides a valuable resource for the community. ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
Author of this paper? Send a correction.