Google Open-Sources Harness² for Recursive Self-Improving Agents

By Jay from Aofeisi QbitAI | Official Account: QbitAI

Google’s latest RSI breakthrough is now open-source.

Named Harness², it enables large models to dynamically modify their own scaffolding code, system prompts, toolchains, and skill libraries during runtime.

Yes, this is RSI at the Harness layer.

The results are impressive.

Equipped with this RSI-specialized GLM-5, performance on the office benchmark WorkBuddy-Bench surged by 11.5 points—

Instantly surpassing native Gemini 3.5 Flash.

In short, it’s an incredibly powerful "cheat code." Any model wrapped in it sees its scores take off immediately.

Currently, the project is fully open-sourced and supports OpenCode and Codex.

Why "Self-Improvement" Often Fails

In the past, the industry has explored various forms of Recursive Self-Improvement (RSI) to make Agents smarter.

The idea sounds appealing: let AI write its own code, modify its own prompts, and optimize its own systems. However, once deployed in real-world open environments, these approaches almost universally fail.

Why?

They primarily get stuck on two critical bottlenecks:

No Ground Truth.

In lab settings, problem-solving has unit tests and Ground Truth to verify correctness. But in real work scenarios—like drafting a business proposal or reviewing a legal contract—there is no one online to grade you.

If an AI modifies its Prompt, how does it know if the change made it smarter or just hallucinated more?

Overfitting to Single Tasks.

After painstakingly modifying the scaffolding to fix one specific task, the next task might be writing a web crawler. The previously modified rules then become burdensome.

Forcing the modified scaffolding onto the next task causes the system to quickly fall into a noise spiral, becoming dumber with every iteration.

The Google team’s new proposal, Harness², specifically targets these two bottlenecks.

They built a Dual-Layer Recursive Architecture.

Dual-Layer Recursion: Experimenting with One Hand, Taking Notes with the Other

The "squared" in Harness² refers to two nested recursive loops: Task-level Recursion and Harnessing-level Recursion.

Layer 1: Task-Level Recursion – Conducting Controlled Experiments Without Ground Truth

Facing a new task, the system first runs a reference trajectory using the initial scaffolding. Regardless of whether the result is correct, the core objective is to identify defects exposed by the model during execution.

Next, it activates a mechanism called Propose–Probe–Compose:

Propose:

A dedicated editor role suggests two sets of modifications based on the previous trajectory.

One set focuses on "Invariant Defect Repair," targeting cross-domain general issues such as chaotic planning, infinite loops in tool calls, or context explosion.

The other set focuses on "Domain-Specific Repair," addressing specific knowledge and constraints relevant to the current business context.

Probe:

Both candidate scaffoldings are run separately on the same task.

Even without a judge, the system compares their execution trajectories against the reference trajectory via Behavioral Contrast. It checks whether the changes resolved the previously stuck steps and whether they introduced new logical regressions.

Compose:

The composer aggregates all execution evidence, selects the modifications proven effective by behavioral evidence, and merges them into the final execution scaffolding.

This isn’t all. It also supports Parallel Recursion, deploying $k$ candidate solutions concurrently for probing, allowing for optimal synthesis within a broader exploration space.

Remarkable.

Layer 2: Harnessing-Level Recursion – Distilling Scaffolding Modification Experience into Meta-Capabilities

Once the current task is completed and deliverables are submitted, many systems end their lifecycle there.

But Harness²’s true strength lies beyond that—

It hands over the entire debugging record (reference trajectories, candidate modifications, comparative evidence, composition decisions) to the Experience Reflector module.

After reflection, the module writes an EXPERIENCE.md, clearly storing why changes were made, under what conditions they took effect, and what consequences they had, into the Harness Improvement Library.

When the next new task arrives:

The execution side still receives a clean, basic scaffolding.

However, the "improvement brain" responsible for modifying the scaffolding already holds a thick manual of historical experience. During the Propose and Compose stages, it can directly retrieve effective components and pitfall avoidance guides from similar past scenarios.

What improves is not just the current scaffolding, but the capability to improve future scaffoldings.

This transforms temporary patches for single tasks into lifelong learning cognitive assets.

Benchmark Results

No matter how elegant the mechanism, it must ultimately speak through data from real-world battlefields.

The team tested on three highly rigorous professional benchmarks:

WorkBuddy-Bench: Covers three high-difficulty office workflows: Code, Web, and Office Suite;

JobBench: Long-horizon, end-to-end complex professional job tasks;

LAB: Harvey’s open-source top-tier legal agent benchmark.

The base models selected were Gemini 3.5 Flash and GLM-5, while the execution scaffoldings chosen were mainstream open-source options OpenCode and Codex.

The results were extremely dramatic.

On WorkBuddy-Bench, with only a single-step Harness² improvement ($k=1$):

GLM-5’s composite score under OpenCode skyrocketed from 64.2 to 75.7, a surge of 11.5 points; under Codex, it jumped from 64.4 to 75.3.

Gemini 3.5 Flash improved from 72.4 to 77.5 under OpenCode, and from 74.5 to 77.5 under Codex.

Look at this data:

GLM-5 with optimized scaffolding (75.7 / 75.3) directly outperformed the original Gemini 3.5 Flash with unmodified scaffolding (72.4 / 74.5).

On the JobBench end-to-end job task benchmark, single-step improvements also yielded comprehensive gains: Gemini 3.5 Flash improved by 4.5 points, and GLM-5 improved by 3.6 points. On the legal benchmark LAB, GLM-5’s partial pass rate surged by 11.8 percentage points.

If the parallel budget is increased to $k=5$ (concurrent execution of 5 candidates), performance improves further: WorkBuddy-Bench scores rise by an additional 3.9 points, JobBench by another 4.2 points, and the LAB pass rate increases by a further 6.7 percentage points.

Even more impressive are the cross-domain transfer tests for the experience library.

The team took the experience library accumulated from running tasks on JobBench and directly applied it to WorkBuddy-Bench. Without any additional probing or fine-tuning, this resulted in seamless score improvements: +0.6 for code, +0.5 for web, and +2.9 for office tasks.

Over the past two years, expectations for AI have been almost entirely pinned on "foundation model scaling." The prevailing belief was that once the next generation of GPT or Gemini arrived, existing application-level challenges would be effortlessly resolved.

However, reality has delivered a harsh wake-up call to all practitioners: foundation models are becoming increasingly expensive, marginal returns are diminishing, and long-tail bugs in real-world deployment scenarios remain endless.

No matter how large you make the context window, the model still cuts corners when reconciling data across files; no matter how high its benchmark scores, it still fails to proactively open files to verify formatting after writing a report.

These issues can never be fully addressed through pre-training parameters alone.

What is needed is a system capable of self-perception, self-correction, and continuous evolution during runtime—a form of Runtime Cognitive Engineering.

Previously, building scaffolding meant engineers sat at their computers manually crafting prompts, configuring tools, and fixing pipelines one by one—an exhausting process prone to leaks and errors.

Now, Harness² demonstrates a new path:

Returning the power of rule-making to the Agent itself.

Paper Authors

Finally, as usual, let’s discuss the paper’s authors.

There are nine authors in total, with a remarkably high proportion of Chinese researchers—reaching 78%.

Due to space constraints, we will introduce three of them in order.

Ruiyao Xu, first author and corresponding author of the paper.

He is a Ph.D. student in the Department of Statistics and Data Science at Northwestern University, advised by Kaize Ding.

His primary research interests include Large Language Models (LLMs), Graph Machine Learning, Anomaly Detection, and AI Agent systems.

This paper represents his work during his internship at Google.

Yanfei Chen, Senior Research Engineer at Google Cloud AI Research.

She earned her undergraduate degree from Peking University, majoring in Electrical Engineering and Economics, and received her Ph.D. in Industrial Engineering from the University of Pittsburgh in 2018.

Subsequently, she served as a Research Engineer at the machine learning company Qeexo. She joined Google in 2020, contributing to projects such as Vertex AI Search and Gemini Enterprise Deep Research. Her main research areas include LLM agents, Retrieval-Augmented Generation (RAG), and tool learning.

Zhongying CuiZhu, also known as Zoey CuiZhu, Research Engineer at Google Cloud AI Research.

She received her Ph.D. in Electrical Engineering from Stanford University in 2019. Her research focuses on general-purpose agents, verifiable agents, and agent quality assessment.

References: [1] https://arxiv.org/html/2610.14224v1