Google ResearchT1·Models
58 pts
Addressing low reproducibility in AI benchmarks caused by ignoring human disagreement, a new study proposes an (N,K) trade-off framework. This model quantifies the resource allocation between 'raters per item' and 'total sample size,' aiming to maximize evaluation stability and credibility within limited budgets.