Rohan Paul (@rohanpaul_ai) · 10/10/2026, 11:05:36
SGUID Method Selects High-Signal Skills to Outperform Full Distillation with Fewer Rules
A joint NYU and Amazon study introduces SGUID, a method that filters skill files based on training signal stability, showing that retaining only skills effective in both early and late training phases (e.g., 6 skills) matches or outperforms full banks of 30-71 skills. Validated on Qwen models for math tasks, adding 3 new skills in a second round boosted Qwen3-8B accuracy from 64.3% to 66.3%, significantly reducing distillation costs while improving efficiency.
SOURCE COVERAGEFull Original Report
Rohan Paul (@rohanpaul_ai) · 2026-10-10 03
Good paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.
