Science's Last Exam Benchmark Grades AI Research Agents With Hidden Expert Checks

1 reports archived1 independent sourcesupdated 11/6/2026, 03:04:36

Synthesis & Latest Updates

1 Sources Cross-Validated

The US nonprofit AItonomy Foundation released Science's Last Exam (SLE), a benchmark spanning 113 tasks across 9 disciplines that hands agents real research goals, data and lab software, then grades them with expert checks the agent never sees, comparing hypotheses against studies published after model training cutoffs. DeepSeek V4.1 Flash led with 21.28% of hypotheses anticipating later research, while all models scored zero on true end-to-end research.

LATEST/The US nonprofit AItonomy Foundation released Science's Last Exam (SLE), a benchmark spanning 113 tasks across 9 disciplines that hands agents real research goals, data and lab software, then grades them with expert checks the agent never sees, comparing hypotheses against studies published after model training cutoffs. DeepSeek V4.1 Flash led with 21.28% of hypotheses anticipating later research, while all models scored zero on true end-to-end research.

Hourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first

Rohan Paul (@rohanpaul_ai)T2Nov 6, 03:04·78 pts
Science's Last Exam Benchmark Grades AI Research Agents With Hidden Expert Checks
- SLE spans 113 tasks in 9 disciplines, with expert-sealed checks hidden from the agent
- Hypotheses are scored against post-cutoff studies so memorization cannot help
- DeepSeek V4.1 Flash led at 21.28% hypothesis hit rate, yet all models scored zero on end-to-end research
View Related Reports & Analysis Original Source