- Modular edit-and-validate lifted success from 5.0% to 40.0% at one-tenth the token cost of full rewrites
- Prompt-only rule edits actually dropped success to 2.30%, so key constraints must live in code
- Self-editing code risks benchmark overfitting and evaluator tampering, requiring validation and permission boundaries
AI CURATED HOT STORY · 11/20/2026
Google's Modular Agent Framework Rewrite Cuts Token Use by 90%
Google researchers have introduced a self-improving agent harness architecture that enables an optimizer model to rewrite framework code module by module, with each change gated through a validation set. On 40 SWE-bench Lite tasks, this modular approach raised success rate from 5.0% to 40.0% while consuming only about 46.7k tokens per task. In contrast, unrestricted full rewrites achieved 42.5% accuracy but required 451.6k tokens per task—nearly 10 times the cost.
The study highlights a critical constraint: embedding rules in prompts caused accuracy to plummet to 2.5%, whereas enforcing constraints at the code level proved effective. Frameworks tuned for one model also tended to stall or waste tokens when transferred to another, indicating model-specific optimization. The research warns of prompt bloat, benchmark overfitting, and evaluation tampering risks associated with self-modifying code, stressing the need for validation mechanisms and permission boundaries.
This work has substantial implications for agent development, demonstrating that modular, validation-driven self-improvement is far more cost-effective than brute-force rewriting. It offers a reusable engineering paradigm for building efficient, transferable agent frameworks, while the flagged risks push the community toward more rigorous evaluation standards and safety boundaries.
Hourly heat curve
4 fully observed hoursThe line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.
Coverage timeline
Total 2 reports · Latest first- IT之家 · 智能时代T2·78 pts
- Rohan Paul (@rohanpaul_ai)T2·78 pts
- Gated single-module edits hit 40.0% accuracy at 46.7k tokens per task; full rewrite reached 42.5% but burned 451.6k tokens
- Enforcing constraints via prompt rules dropped accuracy to 2.5%; rules must be enforced in code
- Harnesses tuned for one model tend to stall or waste tokens on another