- Perplexity (cross-entropy loss) is framed as the core measure of compressor/predictor quality and a shared yardstick across developers
- OpenAI pledges greater pretraining transparency and aims to use the metric to support shared safety standards
- Recent models already show meaningful design contributions from AI, with automation playing a growing role in R&D
AI CURATED HOT STORY · 11/22/2026
OpenAI Chief Scientist Proposes Perplexity as Universal Safety Metric
OpenAI Chief Scientist Jakub Pachocki has proposed using perplexity (cross-entropy loss) as a universal metric for comparing model progress across developers and supporting shared safety standards. He argues that AI progress is fundamentally about training better compressors (predictors), and perplexity directly reflects predictive capability, making it a core measure of pretraining advancement. As a first step, OpenAI disclosed more details about its own pretraining progress, increasing transparency, and noted that AI itself has begun making substantive contributions to model design.
If adopted, the proposal would provide a comparable quantitative benchmark for model capabilities and safety across vendors, helping establish a unified evaluation framework. OpenAI's disclosure also responds to growing calls for AI transparency. However, perplexity as a safety metric remains controversial, as it measures prediction accuracy rather than behavioral compliance or harmfulness.
This move could foster a more open and collaborative industry landscape, encouraging developers to share key metrics and accelerating the co-development of safety standards. Yet its adoption as a universal standard depends on buy-in from other major players and regulatory recognition.
Hourly heat curve
20 fully observed hoursThe line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.
Coverage timeline
Total 2 reports · Latest first- Hacker News AIT2·78 pts
- OpenAI 官方博客T1·78 pts
- Positions perplexity/cross-entropy loss as a comparable cross-vendor metric for pretraining progress
- OpenAI discloses its own pretraining progress as a first transparency step
- Recent models already show meaningful design contributions originating from AI