Today for AI
HOT RADAR
llmHEAT 14.9°

AI CLUSTERED EVENT · 10/14/2026

CMU and Microsoft Release CRBugBench: Claude Opus 5 Catches Only 71.9% of Known Bugs, Confirming Human Review Necessity

1 reports archived1 independent sourcesupdated 10/14/2026, 00:08:50
Synthesis & Latest Updates
1 Sources Cross-Validated

Carnegie Mellon University and Microsoft introduced CRBugBench to evaluate AI's ability to identify real bugs during code reviews. Results show that the top-performing Claude Opus 5 caught only 71.9% of confirmed bugs in its top 5 comments, with performance dropping by up to 25.5% on private repositories, indicating that AI cannot yet fully replace human reviewers.

LATEST/Carnegie Mellon University and Microsoft introduced CRBugBench to evaluate AI's ability to identify real bugs during code reviews. Results show that the top-performing Claude Opus 5 caught only 71.9% of confirmed bugs in its top 5 comments, with performance dropping by up to 25.5% on private repositories, indicating that AI cannot yet fully replace human reviewers.

HEAT TRENDHourly heat curve

4 fully observed hours
Now
14.9
Peak
16.001:00
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

TIMELINECoverage timeline

Total 1 reports · Latest first
  1. Rohan Paul (@rohanpaul_ai)T2·78 pts
    • CRBugBench comprises 1,800 tasks with human-confirmed and fixed real bugs, addressing noise issues from subjective comments in older benchmarks.
    • Claude Opus 5 achieved a 71.9% bug catch rate in its top 5 suggestions, a significant improvement over Opus 4.6 (47.7%) five months prior, but still misses nearly 30% of bugs.
    • In private repository scenarios lacking public context, AI review performance drops by up to 25.5%, highlighting the critical importance of project-specific knowledge for automated reviews.
CMU and Microsoft Release CRBugBench: Claude Opus 5 Catches Only 71.9% of Known Bugs, Confirming Human Review Necessity | Today for AI