Stanford-led Study Finds 27B Open Model Can Self-Improve by Editing Its Own Weights
A new paper from Stanford and other top labs shows a 27B open model can improve itself without training or gradients: Qwen3.5-27B was given access to its own weights, graded practice answers and a Python session with gradients blocked, then reasoned over hidden states for completed versus truncated answers and weakened the direction separating them. On 490 held-out problems fixed in advance, accuracy rose from 57.2% to 63.8%, while the 2B model made no edit and the 4B and 9B models degraded themselves.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-20 17
New Stanford+other top labs paper finds that a 27B open model can improve itself by editing its own weights through reasoning, with no training or gradients.
Improving a deployed model today means another training run on large clusters. Prompts and extra inference compute help, but the gain disappears once the task ends.
They gave Qwen3.5-27B its own weights, its graded practice answers, and a Python session, with gradients blocked. It compared its hidden states on answers that finished against answers that ran out of tokens, then weakened the direction separating them.
Its own edit raised accuracy from 57.2% to 63.8% on 490 unseen problems, in a test fixed in advance. The 2B model made no edit, and the 4B and 9B models broke themselves.
If you run a large open model, let it propose weight edits for its own failures, but keep only edits that pass held-out tests.
