A Stanford study reveals that rapid LLM development has rendered many research findings obsolete. Of 50 widely cited claims tested against 20 models, 44% have 'expired' with an average lifespan of just 15.7 months. Negative claims about what models 'cannot do' expire fastest, highlighting a significant gap between AI capability growth and explanatory research.
Benchmark Saturation
基准饱和
A phenomenon where existing test sets fail to effectively distinguish performance differences among top-tier models.
Curated Intelligence
Artificial Analysis has released a major upgrade to its Intelligence Index (v5), shifting 60% of the evaluation weight to private test sets or held-out answers to combat benchmark saturation. The update introduces two new benchmarks, FrontierCode 2.0 and Terminal-Bench-Science, and launches the Progress Index v1 to track model capability evolution over time.
Mistral AI has officially released its flagship model, Mistral Large 4. Developer Simon Willison conducted a comparative test using the prompt 'armadillo in fishnet tights jaywalking on Mars' to evaluate SVG code generation across frontier models including Claude Opus 5.5 and GPT-6.1. This experiment highlights the new model's practical performance in complex instruction following and multimodal code output.