Rohan Paul (@rohanpaul_ai) · 2026/10/10 11:05:36
NYU 与亚马逊提出 SGUID：筛选高信噪比技能文件，用 6 条规则超越全量蒸馏
纽约大学与亚马逊联合研究提出 SGUID 方法，通过追踪训练信号稳定性筛选“技能文件”，发现仅保留早期和晚期均有效的少数技能（如 6 条）即可匹配或超越包含 30-71 条的全量技能库。该方法在 Qwen 系列模型数学竞赛测试中验证有效，第二轮新增 3 条技能使 Qwen3-8B 准确率从 64.3% 提升至 66.3%，显著降低知识蒸馏成本并提升效率。
报道全文原始报道全文
Rohan Paul (@rohanpaul_ai) · 2026-10-10 03
Good paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.
