ALL INTELLIGENCE
Artificial Analysis Launches v5 Intelligence Index to Combat Benchmark ContaminationInterconnects (Nathan Lambert)T1·Models
Artificial Analysis v5 Released Amid Benchmarking Dilemma
By Nathan LambertOriginal title: Artificial Analysis v5 and the benchmarker's dilemma
45AI Score
Artificial Analysis releases v5, highlighting the challenges in independently evaluating large language models. The article notes that while improving scores is straightforward, creating meaningful benchmarks at the frontier remains difficult.
Full Report
Housekeeping: I’m on the road so the voiceover will be late.
Hill-climbing is easy, knowing what to hill-climb is hard. Making scores go up is expensive, but we have a much clearer playbook for it than we do for creating meaningful benchmarks at the true frontier of AI today.
Independent evaluation of language models is under-appreciated in how hard of a …