Performative Learning Theory

Performative predictions influence the very outcomes they aim to forecast. We study performative predictions that affect a sample (e.g., only existing users of an app) and/or the whole population (e.g., all potential app users). This raises the question of how well models generalize under performativity. For example, how well can we draw insights about new app users based on existing users when both of them react to the app's predictions? We address this question by embedding performative predictions into statistical learning theory. We prove generalization bounds under performative effects on the sample, on the population, and on both. A key intuition behind our proofs is that in the worst case, the population negates predictions, while the sample deceptively fulfills them. We cast such self-negating and self-fulfilling predictions as min-max and min-min risk functionals in Wasserstein space, respectively. Our analysis reveals both a fundamental trade-off between performatively changing the world and learning from it, as well as a surprising insight on how to improve generalization guarantees by retraining on performatively distorted samples. We illustrate our bounds using real data on prediction-informed assignments to job trainings.

Machine learning systems are increasingly used not only to predict the world, but also to change it. A traffic app may change where people drive, and an unemployment-risk model may affect who receives job training, which can then change whether someone remains unemployed. This creates a fundamental difficulty: can we still trust a model trained on data when the data itself changes in response to the model’s predictions? We study this problem mathematically. In particular, we ask when a model trained on a changing sample, such as a test group of users or a group of job seekers, can still make reliable predictions about a larger population that may also react to the model. We prove generalization guarantees for several such feedback settings. Our results show a trade-off: the more strongly predictions change people’s behavior or outcomes, the harder it becomes to learn reliably from the resulting data. We illustrate this trade-off using German labor-market data, where risk predictions can influence access to job-training programs.