Artificial Analysis 更新指数 开源模型差距扩大至 12 分
Artificial Analysis 发布 Intelligence Index v5，引入60%私有测试集以遏制“刷榜”行为。新标准导致 Kimi K3.1 Pro 等开源模型排名大幅下滑，开源与顶级闭源模型的得分差距从3分扩大至12分，凸显了旧基准对开源能力的潜在高估。
报道正文
本文目录2 个章节
New Artificial Analysis index widens the gap between open and closed models
Artificial Analysis overhauled its Intelligence Index, causing Kimi K3.1 Pro to drop five spots in the rankings. The model originally launched in fourth place, but it falls to ninth behind eight closed models in version 5. Artificial Analysis now puts the gap between open models and the top of the leaderboard at 12 points, up from 3 points previously.
Bild: AAII
For the first time, 60 percent of the index relies on private test sets to prevent "benchmaxxing," the practice of training models specifically on known test questions. Kimi K3.1 Pro's drop suggests the old index made some models look better than they were. Grok 4.7 also lost 12 points.
AI researcher Nathan Lambert thinks benchmaxxing is only part of the story. Many training gains today happen in reinforcement learning environments with tasks that resemble the benchmarks. Keeping up across multiple domains at once takes sophisticated RL infrastructure, and not every lab has it. Closed test sets do have a downside, since outsiders can no longer spot errors in the prompts. The AA-Omniscience knowledge test proves those errors exist. Artificial Analysis had to revise ambiguous and flawed questions in that benchmark for the new index.