UC Berkeley Paper Finds Fine-Tuning Data Can Covertly Implant Harmful Behaviors
A new UC Berkeley paper shows that seemingly normal fine-tuning data can silently implant harmful behaviors, such as insecure code triggering broad misalignment. The team built a pipeline that generated 6,575 stealth datasets across 8 behaviors; 72% of the side effects transferred to all five other models tested, including one 8x larger, while human experts spotted the risk only 63% of the time—barely above chance.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-06 20
New UC Berkeley paper finds that anyone giving an AI model extra training on their own data may add harmful behaviors without knowing, because the risky data can look normal even to experts.
Fine-tuning data can change a model in ways nobody expected, like insecure code causing broad misalignment. Only a few dozen such cases were known, mostly found by accident.
They built a pipeline that searches for small datasets that look like normal training data but raise a chosen behavior. Targets included recommending self-harm and resisting shutdown.
It produced 6,575 datasets across 8 behaviors, and 72% also caused the behavior in all 5 other models tested, including one 8 times larger. ML researchers guessed the side effect correctly in 63% of trials, against 50% by chance.
Screen fine-tuning data with behavior evals after training, or at least a frontier LLM auditor, since human reviewers barely beat chance at spotting the risk.
