From the source
Apple researchers have formalized a notion of evaluability for machine learning models that does not assume the test set was held out from training, addressing a gap that is increasingly hard to justify for large foundation models.
- The work asks whether black-box models can be evaluated when test samples may have been seen during training, and studies the problem under different levels of access to the model.
- In the setting where the evaluator only sees the model's output on the test set, the authors fully characterize the distributions where accurate evaluation is possible and give tight upper and lower bounds on the sample complexity of evaluation.
- Their results show that black-box evaluation in this setup is feasible if and only if the distribution is close to being small support.
- The paper then considers evaluation algorithms that can query the model on additional inputs, drawing connections to self-correctors studied in program testing.
- Using these, the authors show that a small amount of query access can increase the power of the evaluator for natural distributions and concept classes.
