Study Finds Monthly Rating Decline for Unchanged AI Models
A study of 1.26 million blind votes on Chatbot Arena found that for identical frozen model pairs, the share of "both are bad" votes rose 1.71 percentage points per month against a 17.1% baseline. Yet in 2024 the stronger model won roughly the same share of decisive votes, indicating rankings held steady and the shift reflects user expectations rather than model capability.
Related Entities
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-13 16
People rate the exact same AI chatbots more harshly each month, so a chatbot that never changes starts to look worse.
They studied 1,260,967 blind votes on Chatbot Arena, where people compare answers from 2 anonymous models. They kept only pairs of frozen models, such as dated snapshots and fixed open weights, so the models never changed.
On the same pair, "both are bad" votes rose 1.71 percentage points a month, on a 17.1% average. Yet in 2024 the stronger model won about the same share of decisive votes, so rankings held.
