Study Finds AI Benchmarks Cover Only 22% of US Job Tasks

1 reports archived1 independent sourcesupdated 10/17/2026, 01:08:38

Synthesis & Latest Updates

1 Sources Cross-Validated

A new 'Benchmark Map' study reveals that existing AI evaluations cover only about 22% of US occupations and a tiny fraction of specific work tasks. By mapping O*NET data against major benchmarks, the research highlights significant blind spots in assessing AI capabilities within real-world economic activities.

LATEST/A new 'Benchmark Map' study reveals that existing AI evaluations cover only about 22% of US occupations and a tiny fraction of specific work tasks. By mapping O*NET data against major benchmarks, the research highlights significant blind spots in assessing AI capabilities within real-world economic activities.

HEAT TRENDHourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

TIMELINECoverage timeline

Total 1 reports · Latest first

Hacker News AIT2·Oct 17, 01:0868 pts
Study Finds AI Benchmarks Cover Only 22% of US Job Tasks
- Only 208 out of 923 US occupations have at least 5 benchmark prompts covering their tasks
- A vast majority of the 18,797 specific work tasks remain uncovered by current AI benchmarks
- Several job families have zero representation in the existing evaluation landscape
View Related Reports & Analysis Original Source