Meta Proposes WM-Basin to Detect Watermark Traces in Distilled LLMs
A new Meta study reveals that models distilled from watermarked API outputs retain hidden, key-specific traces in their weights. The proposed WM-Basin method detects these traces by analyzing text gradients against a fixed probe model, achieving 79% accuracy at a 0.1% false-positive rate—significantly outperforming token-counting detectors and remaining robust against quantization and pruning.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-10-30 08
New Meta paper finds that a model distilled from watermarked API outputs keeps a hidden, key-specific trace in its weights, and that trace can be read from the copy's text alone.
Today's detector counts watermark tokens in the suspect's text and misses most copies made with low-temperature sampling.
They found that watermarked traces push a student's weights in a faint, key-specific direction. WM-Basin turns the suspect's API outputs into gradients of a fixed probe model and tests if they match the real key over decoys.
Across 1,200 distilled models at a 0.1% false-positive rate, it caught 79% with n-gram=1 watermarks, against 29% for token counting. Quantization, light pruning and RL post-training did not remove the trace, though more fine-tuning on clean data weakened it.
– arxiv. org/abs/2610.37266
Title: "Watermark Basins: Black-Box Detection of Distilled LLMs"
