Study Finds Monthly Rating Decline for Unchanged AI Models

1 reports archived1 independent sourcesupdated 11/14/2026, 00:11:25

Synthesis & Latest Updates

1 Sources Cross-Validated

A study of 1.26 million blind votes on Chatbot Arena found that for identical frozen model pairs, the share of "both are bad" votes rose 1.71 percentage points per month against a 17.1% baseline. Yet in 2024 the stronger model won roughly the same share of decisive votes, indicating rankings held steady and the shift reflects user expectations rather than model capability.

LATEST/A study of 1.26 million blind votes on Chatbot Arena found that for identical frozen model pairs, the share of "both are bad" votes rose 1.71 percentage points per month against a 17.1% baseline. Yet in 2024 the stronger model won roughly the same share of decisive votes, indicating rankings held steady and the shift reflects user expectations rather than model capability.

Hourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first

Rohan Paul (@rohanpaul_ai)T2Nov 14, 00:11·71 pts
Study Finds Monthly Rating Decline for Unchanged AI Models
- "Both are bad" votes rose 1.71 percentage points monthly on identical frozen pairs, averaging 17.1%
- In 2024 the stronger model's share of decisive votes stayed roughly constant, so rankings held
- User evaluation standards tighten over time, making static models appear progressively worse
View Related Reports & Analysis Original Source