Google Lets Agents Rewrite Their Own Harness to Boost Task Success Eightfold
Google researchers propose a self-improving harness architecture in which an optimizer model directly edits the agent's runtime framework code, keeping only changes that pass validation. On 40 SWE-bench Lite tasks, the modular approach lifted success from 5.0% to 40.0% at roughly 46.7K tokens per task, far below the 451.6K tokens needed for unconstrained full rewrites. The paper also flags risks of prompt bloat, benchmark overfitting and evaluator tampering.
Full Report
IT之家 November 20 news, a research paper titled "Harnesses That Improve Themselves" newly published by Google researchers on November 17 local time shows that by allowing AI agents to progressively modify their own operating harness and validating each modification, the completion rate of software engineering tasks can be significantly improved.
In a SWE-bench Lite test, the task success rate of an agent using a modular optimization scheme increased from 5.0% to 40.0%, an 8x improvement, with an average consumption of about 46,700 tokens per task.

The research focuses on the AI agent's Harness. It is not the underlying AI model itself, but the operating framework built around the model, responsible for managing context and memory, controlling multi-turn task execution, invoking external tools, and validating execution results before the task ends.
The researchers point out that different models may exhibit明显 differences in tool usage patterns and weaknesses during task execution. Therefore, a Harness optimized for one model may, after being transferred to another model, lead to task stalling, excessive token consumption, or degraded execution performance.
To address this issue, the researchers propose a self-improving Harness architecture. Unlike the traditional approach of constraining agents by modifying prompts, this scheme allows the optimization model to directly modify the Harness code and decide through testing whether to retain the modifications.
The experiment uses an initial coding agent Harness of only 142 lines of code, split into four editable modules, respectively responsible for context and state management, agent control, tool invocation interfaces, and validation and failure recovery.
In each optimization round, the system allows modification of only one module. The optimization model first identifies problems based on the agent's execution records and proposes code modification plans, then checks the modification effects through a validation set, while being constrained by a token usage cap. Only modifications that pass validation are retained, and plans that fail to meet the requirements are discarded.
In the core test, the researchers used Gemini 3.8 Flash as the base model for executing tasks and evaluated the performance of different schemes on 40 SWE-bench Lite software engineering tasks that were not involved in optimization.
The results show that the initial Harness had a task success rate of 5.0%. After adopting the modular modification and validation mechanism, the success rate increased to 40.0%, with an average consumption of about 46,700 tokens per task.
As a control, if the optimization model is allowed to rewrite the Harness as a whole without restrictions, the task success rate can reach 42.5%, slightly higher than the modular scheme, but it requires an average consumption of about 451,600 tokens per task, nearly 10 times that of the modular scheme.
IT之家 notes that another experiment constrained the agent by modifying only the prompt rule file, and the task success rate instead dropped from the initial 5.0% to 2.5%.
The study also examined the performance of the optimized Harness when transferred to other models. Without re-running the optimization search, the modular scheme achieved a task success rate of 37.5% after being transferred to Gemini 4 Argon, while the whole-rewrite scheme achieved 30.0%.
The researchers believe that self-improvement of the Harness should not rely solely on prompts, but should implement important constraints in actual code. For example, automatically testing and checking task execution results, identifying repeated failures, and preventing unverified task completion can enable agents to complete complex tasks more reliably.
The paper also points out that allowing agents to modify their own operating code introduces new risks, including continuously expanding prompt content, over-optimization for benchmarks, and tampering with evaluation programs. Therefore, code modifications need to be constrained by validation mechanisms and permission boundaries.
It should be noted that the experiments in this study mainly focus on specific models and SWE-bench Lite software engineering tasks, and the core test set contains only 40 tasks that were not involved in optimization, which is not yet sufficient to prove that this method can achieve the same results across all models and agent application scenarios.