OpenAI Chief Scientist Jakub Pachocki argues that progress in modern AI is fundamentally about training better compressors, i.e. predictors, and proposes perplexity (equivalently cross-entropy loss) as a comparable metric across model developers to underpin shared safety standards. He also pledges more transparency into OpenAI's pretraining progress and notes that recent models already show meaningful design contributions from AI.
Termsslug: perplexity
Perplexity
困惑度
A measure of a language model's predictive uncertainty; lower values indicate stronger compression and prediction of data.
Curated Intelligence
Hacker News AIT2·Models
78 pts
Hugging Face BlogT1·Models
68 pts
OpenMed released an end-to-end protein AI pipeline and open-sourced cross-species mRNA language models based on CodonRoBERTa-large-v2. Trained across 25 species in 55 GPU-hours for just $165, the model outperforms ModernBERT with a perplexity of 4.10 and higher Spearman CAI correlation.