ReTerm Enables Recursive Self-Improvement for Code Agents via Failure Recovery

What does a Code Agent do next when it writes buggy code, encounters command errors, or breaks the environment?

In complex terminal tasks, a single error often triggers a cascade of invalid operations: repeatedly installing dependencies that are already present, continuing to modify code based on incorrect assumptions, and eventually exhausting the budget before failing the task. Can these failures be repurposed as training material for the Agent's self-improvement?

Researchers from Fudan University, MMLab at The Chinese University of Hong Kong, and Shanghai Jiao Tong University have proposed ReTerm. This approach enables terminal agents to attempt recovery from their own failures and learn from the outcomes, exploring a path of Recursive Self-Improvement (RSI) driven by their own errors.

The key insight of this work is integrating "encountering problems, attempting self-recovery, and updating parameters based on success or failure" into a single training loop. As the model evolves, the nature of its failures changes, and these new failures become the starting point for the next round of learning.

• Paper Link: https://arxiv.org/abs/2610.18484
• Code, Models & Data: https://github.com/twothsnd/ReTerm
• Hugging Face Paper: https://huggingface.co/papers/2610.18484

Why Agents That Perform Tasks Still Get Stuck in Errors

Error correction in terminal environments is more complex than fixing an answer to a reasoning problem. Erroneous operations may have already modified files, installed dependencies, or started processes; subsequent actions face an environment altered by previous steps. To continue completing the task, the Agent must understand what has happened and diagnose exactly where things went wrong.

A PyStan case study in the paper illustrates this clearly: although the package was installed, executing import pystan still resulted in an error. A model trained with standard GRPO repeatedly attempted to import, uninstall, and reinstall, never breaking out of the erroneous direction. In contrast, a model trained with ReTerm-GRPO checked the installation path, discovered that the actual module resided in the stan directory, and successfully recovered by using import stan.

How can such recovery behavior be made more stable? Learning solely from successful trajectories in clean environments makes it difficult to adequately cover the complex states resulting from real-world failures. Furthermore, artificially injected errors may not align with the mistakes models actually make. ReTerm places the starting point of training in the failure scenarios left behind by the Agent after executing tasks itself.

ReTerm-10K: Teaching the Model to Take Over Failure Scenarios First

The team constructed ReTerm-10K to provide recovery demonstrations for the model. They first collected tasks failed by the base model, then had a stronger teacher model take over, demonstrating how to inspect issues, fix errors, and complete the tasks.

After validation for success and quality filtering, the team obtained approximately 10,000 recovery trajectories to support cold-start training. The model first learns to recover from demonstrations, then explores how to solve newly encountered problems through online reinforcement learning.

ReTerm-GRPO: Feeding Its Own Failures into the Next Training Round

The idea behind ReTerm-GRPO is straightforward: let the Agent attempt to fix its own errors, then learn from the results of those fixes.

When the Agent fails a task, the system preserves the environment state and operation logs at that moment. Starting from the same failure scenario, the Agent independently attempts various recovery strategies. A task verifier determines which attempts ultimately completed the task.

The success or failure of these recovery attempts is converted into reinforcement learning signals to update the model parameters, teaching the Agent to diagnose problems and fix errors more effectively.

Self-improvement occurs within this loop: the model continuously encounters its own problems, autonomously attempts recovery, and learns from the results. As capabilities evolve, newly generated failures enter the training process, driving simultaneous improvements in both recovery ability and task-solving capability.

9B Model Achieves Simultaneous Gains Across Three Major Benchmarks

Across three major benchmarks covering terminal operations and software engineering, ReTerm-9B achieved improvements of over 13 percentage points compared to the base model:

• Terminal-Bench 2.0: 20.2% → 34.8%, an improvement of 14.6 percentage points.
• SWE-bench Verified: 45.2% → 58.4%, an improvement of 13.2 percentage points.
• OpenThoughts-TBLite: 44.2% → 60.6%, an improvement of 16.4 percentage points.

Even 9B models can demonstrate strong task-solving capabilities. Across three major benchmarks, ReTerm-9B achieved significant improvements, showing competitive performance among same-scale models and outperforming larger-parameter models on certain evaluations. This indicates that learning to recover from one's own failures is an effective path for enhancing terminal agent capabilities.

Note that ReTerm and Qwen3.5-9B use the same runtime framework and evaluation configuration, allowing for a direct observation of gains from recovery training; data for other models comes from their respective papers or official releases, where runtime configurations may differ.

Effectiveness Persists Across Different Runtime Frameworks and Task Types

Performance gains persist even when switching runtime frameworks. On the same Terminal-Bench 2.0 tasks, ReTerm outperformed the base model across three different runtime frameworks, retaining a gain of 6.7–12.4 percentage points. This demonstrates that these benefits are transferable to different interaction settings.

The benefits of recovery training also extend to mathematical reasoning and spreadsheet manipulation. Pass@1 on AIME 2025 improved from 73.3% to 93.3%, and SpreadsheetBench Verified scores rose from 37.8% to 48.8%. Improvements were also observed on AIME 2024 and Aider Polyglot, showcasing potential for cross-task generalization.

Improvements Stem from More Effective Recovery

Ablation studies further address a key question: What exactly does explicitly training recovery capabilities offer beyond standard reinforcement learning?

Ablation experiments further validated the role of recovery training. On Terminal-Bench 2.0, SFT with recovery demonstrations improved performance from 20.2% to 25.8%, while standard GRPO reached 28.0%. ReTerm-GRPO achieved 34.8%. Building upon standard reinforcement learning, specifically optimizing recovery behavior yielded an additional 6.8 percentage point gain.

As task completion rates increased, agents also became better at recovering from errors. For five categories of errors—planning, tool usage, code implementation, verification, and resource management—the recovery rate of ReTerm-GRPO was higher than both the base model and standard GRPO, indicating that the benefits of recovery training manifest across various execution scenarios.

These results suggest that ReTerm is more likely to adjust its course after making mistakes and continue completing tasks. However, except for tool usage, the recovery rates for the other four error categories remain below 32%, leaving room for future improvement.

Turning Failures into Starting Points for Next-Round Self-Improvement

ReTerm presents a concrete pathway for agent self-improvement: allowing the model to retain the "scene" of its own failures, autonomously attempting recovery, and then using verifiable task outcomes to train the next policy iteration. Failure transforms from being the end of an execution episode into material for subsequent learning.

This work provides an experimental and analyzable entry point for failure-driven Recursive Self-Improvement (RSI). As policies evolve, models can continuously face new problems they generate themselves, allowing recovery capabilities and task-solving abilities to develop jointly within the same training process.

The research team has open-sourced the code, models, and data. We welcome your interest and engagement.

© THE END

Please contact this official account for authorization before reprinting.

For submissions or media inquiries: [email protected]

Open in WeChat