Rohan Paul (@rohanpaul_ai) · 10/11/2026, 04:40:32
Stanford Study: Fix Agent Loops via Harness Edits, Poor Plans via Weight Fine-Tuning
A Stanford study categorizes agent failures into process errors (loops, budget exhaustion) fixable by harness edits, and content errors (poor planning) requiring weight fine-tuning. Experiments show harness evolution improved Qwen3.5-4B scores from 0.16 to 0.30 but failed to reduce bad plans, whereas a LoRA adapter cut poor plan rates in Qwen3.5-9B from 28% to 5%. This establishes a diagnostic framework for optimizing agents.
SOURCE COVERAGEFull Original Report
Rohan Paul (@rohanpaul_ai) · 2026-10-10 20
New Stanford paper finds that harness changes fix agents that loop or stall, while agents that deliver bad plans need weight training instead.
An agent can be improved by editing its harness, the prompts, tools, and checks around the model, or by fine-tuning its weights.
They sorted failed runs into process failures, such as loops and used-up step budgets, and content failures, where a poor plan was delivered. On a travel-planning benchmark, an LLM-driven loop rewrote the harness, and its best runs then fine-tuned the model.
Harness evolution lifted Qwen3.5-4B from 0.16 to 0.30 on held-out tasks, as plan delivery rose from 55% to 90%. Harness edits never shrank the share of poor plans, but a LoRA adapter cut them from 28% to 5% of Qwen3.5-9B's held-out runs.
Before improving an agent, label why its runs fail, then fix process failures in the harness and content failures in the weights.
