Prediction without Preclusion Recourse Verification with Reachable Sets

Machine learning models are often used to decidewho will receive a loan, a job interview, or a pub-lic service. Standard techniques to build thesemodels use features that characterize people butoverlook their actionability. In domains like lend-ing and hiring, models can assign predictions thatare fixed – meaning that consumers who are de-nied loans and interviews are permanently lockedout from access to credit and employment. In thiswork, we introduce a formal testing procedure toflag models that assign these “predictions with-out recourse," called recourse verification. Wedevelop machinery to reliably test the feasibilityof recourse for any model given user-specified ac-tionability constraints. We take a data centric ap-proach to demonstrate how these tools can ensurerecourse and adversarial robustness in real-worlddatasets and use them to study the infeasibilityof recourse in real-world lending datasets. Ourresults highlight how models can inadvertently as-sign fixed predictions that permanently bar accessand the need to design algorithms that account foractionability when developing models and provid-ing recourse.