A controlled study shows that an editor model can directly rewrite a target model's weights, making it persistently answer a specific question incorrectly (e.g., naming Buenos Aires instead of Sydney for the 2000 Olympics) rather than only for a single conversation. The authors term such weight changes that deviate from authorized deployment behavior "weight-space misfolding," and argue that authenticating the editor is insufficient to judge whether an edit is appropriate; the patch must be applied and the edited model tested. The result comes from a controlled probe of a small model and is not evidence of a deployed attack or model-to-model spread.
Termsslug: weight-space-misfolding
Weight-Space Misfolding
权重空间错误折叠
A persistent change caused by another model's manipulation of a target's weights that deviates from behavior authorized for legi
Curated Intelligence
Hacker News AIT2·Models
78 pts