仅改 0.016% 权重即可消除模型推理死循环

原标题:Show HN: Watch weight-space programming in action with WebGPU
74AI 研判分

作者用 WebGPU 构建了一个交互式演示,展示“权重空间编程”:无需梯度下降,仅修改 Qwen 小模型约 80 亿参数中 0.016%(13.1 万)的 ffn_down 输出权重,就显著缓解了模型反复自我验证、耗尽 token 预算的死循环问题。同一模型同一提示下,触及 32k token 上限的运行比例从 93.0% 降至 33.1%,MATH-500 准确率从 7.9% 升至 48.1%。该方法定位少数具体失败模式及其对应神经元,与模型汤(souping)的广谱合并形成对比,并被视为智能体“提议-测试-反馈”闭环的潜在应用方向。

报道正文

本文目录4 个章节

The base model gets stuck in a loop

With thinking on, small Qwen keeps re-checking itself — “wait, let me verify…” — until it runs out of budget instead of answering.

93.0% Runs that hit the 32k-token limit 2

An LLM edits 0.016% of the weights

It subtracts one direction from 64 neurons' output weights (ffn_down) in layers 15 and 19 — 131,072 of ~800M weights.

layer 0layer 23

Stripes drawn ~25× wider than true scale — at real size they would be invisible.

3

The edited model escapes

Same model, same prompts — it now stops looping and finishes its answers.

Hit the token limit 93.0%→33.1% MATH-500 accuracy 7.9%→48.1%

Where this might fit

Model souping - diffusion-LoRA merging is one case of it - is weight-space editing too. The difference is the target: souping picks merge weights (and sometimes merge order) with a broad benchmark; weight-space programming zeroes in on a few specific failure cases and the neurons behind them. And here's the fun part: it's a promising fit for agents - propose an edit, test it, feed the result back - the same edit-test-repeat loop behind the recent coding-agent wins.


作者发帖说明(HN):

I wanted to build a full weight-space programming loop entirely in the browser with WebGPU: let an LLM inspect Qwen3.5-0.8B, edit its weights, evaluate, and repeat.

the full loop needed too much VRAM. so I turned it into an interactive walkthrough of a concrete example: editing 0.016% of a model's weights to reduce reasoning loops, no gradient descent.

Based on exp8 of “Sparks of Weight-Space Programming.”: https://news.ycombinator.com/item?id=50341174