Google Study Finds Gated Modular Edits Cut Agent Harness Token Cost by 90%

Original title: New Google paper finds that an unchecked full rewrite of an agent harness matched accuracy but cost ...
78AI Score

A new Google study finds that self-improving agent harnesses work best when an optimizer model edits one code module at a time under validation and token caps, rather than performing an unchecked full rewrite. On 40 held-out SWE-bench Lite tasks, the gated approach reached 40.0% accuracy at 46.7k tokens per task, versus 42.5% at 451.6k tokens for a full rewrite, nearly 10x the cost. Enforcing rules via prompts instead of code collapsed accuracy to 2.5%.

Full Report

Rohan Paul (@rohanpaul_ai) · 2026-11-19 21

New Google paper finds that an unchecked full rewrite of an agent harness matched accuracy but cost nearly 10 times more tokens per task than gated modular edits.

that self-improving agent harnesses work best when they change 1 code module at a time and must pass validation.

The harness is the code around a model that manages context, runs tools, and checks work before stopping. A harness tuned for 1 model often stalls or wastes tokens on another.

They let an optimizer model improve a 142-line coding harness by editing 1 of its 4 parts at a time. Each edit had to pass a validation set and a token cap, or it was rejected.

On 40 held-out SWE-bench Lite tasks with Gemini 3.8 Flash, the gated harness solved 40.0% against 5.0% for the seed, using 46.7k tokens per task. A full unchecked rewrite solved 42.5% but used 451.6k tokens, and prompt rules dropped accuracy to 2.5%.

If you let agents tune their harness, gate each change on held-out tasks and token cost, and enforce rules in code, not prompts.

推文配图