Microsoft and UCSB Propose ScholarEvolve for Self-Evolving Agents

Can AI Agents Learn from Research Papers Like Human Researchers?

Over the past two years, the capabilities of Large Language Models (LLMs) have improved significantly, enabling AI Agents built around these models to handle increasingly complex tasks. From calling tools and operating software to planning and executing multi-step workflows, agents have evolved from simple Q&A systems into entities capable of independently solving long-horizon tasks involving complex tool invocations. Simultaneously, a vast number of new preprints in the agent domain are published daily on arXiv.

However, a practical challenge has emerged: How can we improve an Agent when it hits a capability bottleneck?

The most direct approach is training stronger models. Yet, agent performance does not depend solely on the underlying model. Factors such as how tools are invoked, how context is managed, how experiences are reused, and how tasks are planned all influence final execution outcomes.

These software systems surrounding the model are typically referred to as the Agent Harness.

Recently, research efforts like Meta Harness have begun exploring automated harness optimization: having a Coding Agent analyze failures during task execution, attempt to modify the Harness code, and experimentally search for better versions.

While this approach shows potential, it has a clear limitation: improvements proposed by the Coding Agent rely heavily on the existing knowledge of its backbone model and previously observed failure cases. If a problem hasn't occurred yet, or if the model itself doesn't know a better solution, it struggles to proactively explore relevant improvement directions.

When human researchers face similar issues, they typically read the latest papers, understand methods proposed by other researchers, and then try applying those methods to their own systems. Can we equip Agents with this same capability?

Recently, a research team from UCSB and Microsoft introduced ScholarEvolve, which attempts to automate the research process of reading literature, implementing algorithms, and evaluating effectiveness, allowing the latest academic findings to directly participate in the evolution of the Agent Harness.

• Paper: Learning from Research: Toward Lifelong Agent Harness Evolution
• Paper Link: https://arxiv.org/abs/2609.40169
• Code Repository: https://github.com/UCSB-NLP-Chang/ScholarEvolve
• Institutions: University of California, Santa Barbara (UCSB), Microsoft

Without modifying the parameters of the underlying model, ScholarEvolve increased the task completion rate of Qwen3.5-27B on the AppWorld Challenge from 49.6% to 63.6%, an improvement of 14.0 percentage points.

ScholarEvolve: Sustainable Agent Evolution via Automated Research

The core idea behind ScholarEvolve is straightforward: instead of relying solely on its own experience to find improvements, the Agent actively acquires new methods from research literature. However, reading papers is only the first step. The real difficulty lies in identifying diverse improvement directions from a massive body of research and translating them into a Harness that functions effectively when integrated.

To address this, ScholarEvolve designs an automated research and evolution pipeline.

1) Decompose the Agent Harness into five primary functional modules.

These modules are responsible for Tools, Context Management, Skills, Memory, and Workflow. This decomposition allows the system to target improvements more precisely. For example, if an Agent frequently loses historical information, the system can investigate new context compression or memory retrieval methods. If an Agent fails to complete long-horizon tasks effectively, it can explore new task planning and workflow designs.

2) Have the Research Agent find solutions from papers.

The system first analyzes the Agent's execution traces to identify recurring problems and translates them into generalized capability requirements.

Subsequently, the Research Agent constructs retrieval queries based on the responsibilities of different modules, searching for relevant studies in academic sources like arXiv.

However, ScholarEvolve does not simply select the most highly relevant papers from the search results. Highly relevant papers often use very similar methods. If the limited experimental budget were spent entirely on these papers, the search would easily become confined to a few technical pathways. To solve this, ScholarEvolve uses Topic Modeling to categorize retrieved papers by distinct technical mechanisms, aiming to cover mutually different research directions.

For instance, regarding context management improvements, some papers reduce length through context compression, while others preserve key evidence by retrieving important fragments. Although they address similar problems, they employ different mechanisms, enabling ScholarEvolve to explore a wider variety of solutions within a limited budget.

3) Implement paper methods as code and find the optimal combination.

The Research Agent further reads the selected papers, including GitHub code implementations and prompt details in appendices, generates specific implementation plans, and hands them over to the Coding Agent to modify the corresponding Harness modules. Each candidate implementation is tested for actual effectiveness on validation tasks. However, good performance of a single module does not guarantee effectiveness when combined with other modules.

Therefore, ScholarEvolve also attempts to combine improvements across different modules, evaluates the performance of the complete Harness, and retains verified designs that outperform the current version.

The entire process resembles automated scientific research: extracting core methods from the latest literature, implementing them in code, and deciding whether to adopt them based on experimental results.

How much can Agent performance improve by evolving only the Harness?

The research team conducted experiments on two representative Agent Benchmarks: AppWorld and τ²-Bench.

• AppWorld requires Agents to operate different applications via APIs to complete real-world tasks involving multiple steps;
• τ²-Bench focuses on task execution capabilities that require tool calling and user interaction.

The experiments used Qwen3.5-27B and GPT-5.4-mini as Backbone models, comparing them against the original Harness and the Meta Harness.

On AppWorld Normal, the Qwen3.5-27B optimized by ScholarEvolve saw its task completion rate increase from 69.0% to 81.4%.

Notably, in the AppWorld Challenge, the test set includes applications such as Amazon and Gmail that did not appear during the training and validation phases, thereby further testing whether the improved Harness possesses generalization capabilities. Under this setting, the task completion rate for Qwen3.5-27B increased from 49.6% to 63.6%. In contrast, the Meta Harness under the same experimental setup only reached 54.6%. For GPT-5.4-mini, ScholarEvolve also achieved significant improvements, raising the task completion rate from 46.1% to 55.6% on the AppWorld Challenge.

On τ²-Bench Telecom, the single-task success rate for GPT-5.4-mini also improved from 72.7% to 81.9%. These results demonstrate that even without additional model training, simply optimizing the Harness can significantly improve the actual performance of Agents.

The research team also conducted ablation studies to examine the roles of literature guidance, modular search, and Topic Modeling.

The results showed that removing these designs led to a gradual decline in Agent performance. This indicates that the performance gains stem not only from automatic code modifications but also from how different research methods are found, organized, and combined.

Another interesting finding is that the individually best-performing modules do not necessarily compose the best-performing Harness. This further suggests that Harness improvement cannot focus solely on one component; it must also consider the interactions between different modules during actual execution.

From One-Time Optimization to Lifelong Continuous Learning and Evolution

A more ambitious goal of ScholarEvolve is to enable Agents to continuously improve alongside advancements in Agent research. Traditional Harness optimization typically converges after several rounds of search, resulting in a fixed version of the Agent. However, the research field itself is constantly evolving. New tool-calling methods, memory architectures, or reasoning strategies may emerge months later.

ScholarEvolve aims to leverage these emerging research findings, allowing Agents to proactively try new improvement directions even if they haven't encountered new failures.

To validate this idea, the research team constructed three consecutive research windows based on paper publication dates (December 2025, January–April 2026, and May–August 2026), simulating the Harness evolution process as new papers were gradually added while keeping the underlying model unchanged.

Experimental results showed that the task completion rate for Qwen3.5-27B on AppWorld Normal progressed from an initial 69.0%, through multiple rounds of evolution, to 73.4%, then 79.8%, and finally reaching 81.5%.

Integrating Research Findings into Agent Systems

ScholarEvolve does not imply that Agents have become fully autonomous in solving all their own problems.

Further analysis in the paper revealed that although the evolved Harness significantly reduced many issues, there remains room for improvement in areas such as constraint tracking for complex tasks and the completeness of information retrieval.

Methods extracted from papers are not always effective. Some candidate implementations even led to performance degradation, making actual evaluation and filtering indispensable.

However, ScholarEvolve explores a noteworthy path. Previously, after a new Agent paper was published, researchers or engineers typically needed to read, understand, reproduce, and manually integrate it into existing systems before it could generate real value. ScholarEvolve attempts to connect this process, enabling AI Agents to actively read research, implement new mechanisms, and verify their utility through experiments.

As related research continues to advance, future Agents may not only complete tasks assigned by users but also automatically and continuously learn from new knowledge.

Author Introduction

Jingbo Yang is a Ph.D. student in the Department of Computer Science at the University of California, Santa Barbara (UCSB). His research interests include Large Language Model agents, self-evolving systems, agent memory, and efficient inference. He has completed research internships at Microsoft, with relevant findings published in international academic conferences such as NeurIPS and ACL.

© THE END

Please contact this official account for authorization to reprint.

For submissions or media inquiries: [email protected]