Abstract

Learning and Evaluating Classifiers under Sample Selection Bias
Bianca Zadrozny - IBM T.J. Watson Research Center
Classifier learning methods commonly assume that the training data consist ofrandomly drawn examples from the same distribution as the test examples aboutwhich the learned model is expected to make predictions. In many practical situations, however, this assumption isviolated, in a problem known in econometrics as sample selection bias. In thispaper, we formalize the sample selection bias problem in machine learning terms and study analytically and experimentally how anumber of well-known classifier learning methods are affected by it. We alsopresent a bias correction method that is particularly useful for classifierevaluation under sample selection bias.

Learning and Evaluating Classifiers under Sample Selection Bias

Bianca Zadrozny - IBM T.J. Watson Research Center

Classifier learning methods commonly assume that the training data consist ofrandomly drawn examples from the same distribution as the test examples aboutwhich the learned model is expected to make predictions. In many practical situations, however, this assumption isviolated, in a problem known in econometrics as sample selection bias. In thispaper, we formalize the sample selection bias problem in machine learning terms and study analytically and experimentally how anumber of well-known classifier learning methods are affected by it. We alsopresent a bias correction method that is particularly useful for classifierevaluation under sample selection bias.