Disparate Conditional Prediction in Multiclass Classifiers

We propose methods for auditing multiclass classifiers for fairness under multiclass equalized odds, by estimating the deviation from equalized odds when the classifier is not completely fair. We generalize to multiclass classifiers the measure of Disparate Conditional Prediction (DCP), originally suggested by Sabato & Yom-Tov (2020) for binary classifiers. DCP is defined as the fraction of the population for which the classifier predicts with conditional prediction probabilities that differ from the closest common baseline. We provide new local-optimization methods for estimating the multiclass DCP under two different regimes, one in which the conditional confusion matrices for each protected sub-population are known, and one in which these cannot be estimated, for instance, because the classifier is inaccessible orbecause good-quality individual-level data is not available. These methods can be used to detect classifiers that likely treat a significant fraction of the population unfairly. Experiments demonstrate the accuracy of the methods. The code for the experiments is provided as supplementary material.

Many machine learning systems make decisions that affect people's lives, like approving loans or recommending medical treatments. When access to the underlying system is difficult, it becomes harder to check if they are treating all groups of people fairly. Moreover, existing fairness checks often do not address cases where these systems handle more than two possible outcomes. We introduce new methods to audit these multiclass decision systems for fairness. We build on a fairness measure called Disparate Conditional Prediction (DCP), which looks at how many people receive predictions that differ from a fair baseline. We extend this measure to work with systems that support more than two outcomes, and provide two ways to estimate the DCP, one for cases in which we have detailed data about how the system behaves for different groups, and the other for cases when we do not have access to the system or high-quality individual data. These tools make it easier to detect when a decision-making system is likely treating a significant portion of the population unfairly and thus helps organizations and regulators identify and address bias leading to fairer outcomes for everyone.