Apple Machine Learning ResearchT1·Models
58 pts
The paper formalizes a notion of evaluability that drops the classical held-out test set assumption, asking whether black-box models can still be evaluated when test samples may have appeared in training. The authors fully characterize the distribution classes admitting accurate evaluation and prove tight upper and lower bounds on sample complexity, showing black-box evaluation is feasible if and only if the distribution is close to a specific condition.