Understanding Fixed Predictions via Confined Regions

Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Existing approaches to audit fixed predictions do so on a pointwise basis, which requires access to an existing dataset of individuals and may fail to anticipate fixed predictions in out-of-sample data. This work presents a new paradigm to identify fixed predictions by finding confined regions of the feature space in which all individuals receive fixed predictions. This paradigm enables the certification of recourse for out-of-sample data, works in settings without representative datasets, and provides interpretable descriptions of individuals with fixed predictions. We develop a fast method to discover confined regions for linear classifiers using mixed-integer quadratically constrained programming. We conduct a comprehensive empirical study of confined regions across diverse applications. Our results highlight that existing pointwise verification methods fail to anticipate future individuals with fixed predictions, while our method both identifies them and provides an interpretable description.

Some machine learning models make predictions that are “fixed”—meaning no matter how a person changes their situation, the model’s decision stays the same. This can be a big problem if someone is denied a loan, job, or opportunity and has no way to improve their chances. Current tools to detect these fixed decisions work on a person-by-person basis and can miss new or unusual cases. This means that we may not know our model could deny individuals opportunities with no way to rectify them until after we’ve started using it! Our research introduces a new way to find groups of people who will get a fixed prediction. This works even when we don’t have much data and can help practitioners explain why certain people are stuck with a fixed outcome. We also show that our method finds problems that other tools miss—and it does so in a way that’s fast and easy to understand.