Nous Research has open-sourced Arete-27B, a Qwen3.8-27B fine-tune trained with plain supervised fine-tuning on 52.8M tokens using Inkling as teacher, with no RL or architecture changes. On its self-built 803-prompt eval, propaganda-style answers dropped from 22% to 0.9%, while factuality mean rose only from 2.32 to 2.98 out of 5 and still trails Muse Glimmer 30B; 216 of 803 answers still score 0, and both eval and training data are self-constructed and self-judged.
Termsslug: teacher-model
Teacher Model
教师模型
A model that produces training samples or soft labels to guide the training of a target model in distillation or data-generation
Curated Intelligence
Marktechpost (@Marktechpost)T2·Open Source
72 pts
Rohan Paul (@rohanpaul_ai)T2·Models
74 pts
A new Salesforce paper compares on-policy distillation against GRPO using held-out accuracy per training FLOP. It finds that distilling from a large external post-trained teacher generally costs more compute than plain RL and tends to inflate response length, whereas training per-domain RL experts and distilling them into a single student matches joint GRPO on math and code with 42% fewer total FLOPs.