约翰霍普金斯研究：训练中只出现一次的事实越多，大模型科学发现能力越强
约翰霍普金斯大学新论文发现，训练数据中仅出现一次的事实（monofacts）占比从 0% 升至 100% 时，模型在材料与 DNA 序列上的科学发现率持续上升，该趋势在两个模型家族中均成立。作者将幻觉理论延伸至发现能力：发现与幻觉本质都是生成训练中未见的内容，因此保留稀有单次样本、避免重复数据可提升发现能力，但也会带来更多幻觉。
报道正文
Rohan Paul (@rohanpaul_ai) · 2026-11-05 22
Strange findings here.
New Johns Hopkins paper finds that LLMs make more scientific discoveries when more of their training facts appear only once.
discovery rates rose steadily as the share of once-seen facts in training went from 0% to 100%, for 2 model families.
Discovery and hallucination both mean producing something the model never saw in training. Earlier theory tied hallucination to "monofacts", facts seen exactly once, and treated that unseen space as mostly false.
The authors extend that theory to show monofacts also set how much a model can discover. They fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, varying the monofact share from 0% to 100%.
Discovery, scored as exactly recovering materials or DNA sequences held out from training, rose steadily with the monofact share in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.
When building science models, keep rare one-off examples and avoid repeating data, and expect more hallucinations to come with more discoveries.
