Termsslug: test-set-contamination

Test Set Contamination

测试集污染

Phenomenon where evaluation data was seen during training, distorting benchmark results.

Curated Intelligence

1 items · Page 1/1
Apple Machine Learning ResearchT1·Models
58 pts
Researchers Formalize Evaluability Theory for Contaminated Test Sets

The paper formalizes a notion of evaluability that drops the classical held-out test set assumption, asking whether black-box models can still be evaluated when test samples may have appeared in training. The authors fully characterize the distribution classes admitting accurate evaluation and prove tight upper and lower bounds on sample complexity, showing black-box evaluation is feasible if and only if the distribution is close to a specific condition.