Johns Hopkins Study Links Rare Training Facts to Stronger LLM Discovery
A new Johns Hopkins paper finds that LLM discovery rates rise steadily as the share of once-seen training facts (monofacts) increases from 0% to 100%, across two model families. The authors extend hallucination theory to discovery: both involve generating content unseen in training, so retaining rare one-off examples and avoiding data repetition boosts discovery but also increases hallucinations.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-05 22
Strange findings here.
New Johns Hopkins paper finds that LLMs make more scientific discoveries when more of their training facts appear only once.
discovery rates rose steadily as the share of once-seen facts in training went from 0% to 100%, for 2 model families.
Discovery and hallucination both mean producing something the model never saw in training. Earlier theory tied hallucination to "monofacts", facts seen exactly once, and treated that unseen space as mostly false.
The authors extend that theory to show monofacts also set how much a model can discover. They fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, varying the monofact share from 0% to 100%.
Discovery, scored as exactly recovering materials or DNA sequences held out from training, rose steadily with the monofact share in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.
When building science models, keep rare one-off examples and avoid repeating data, and expect more hallucinations to come with more discoveries.
