Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning

Differentiable optimization layers are traditionally integrated in predict-then-optimize frameworks where a neural model estimates parameters that subsequently serve as fixed inputs to downstream decision-making optimization problems. In this work, we introduce the concept of a "fairness layer": a differentiable optimization layer appended to a model's output layer that guarantees a chosen notion of output parity is satisfied when integrated into a neural network. Additionally, we introduce an online primal-dual inference algorithm that provides provable aggregate fairness guarantees for streaming predictions with arbitrarily small batch sizes, where traditional per-batch constraints become overly restrictive. Numerical experiments demonstrate the effectiveness of the fairness layer and associated algorithm, and theoretical analysis characterizes the layer's differentiability and stability properties during model training and backpropagation. Our code for these experiments is publicly available on GitHub: https://github.com/dtroxell19/FairDL-ICML-2026.git and our public Python package documentation can be found online: https://dtroxell19.github.io/fairness_training/.

AI systems are being integrated into society in ways that affect many people, such as lending decisions, hiring decisions, and AI-enabled medical devices. As a result, many laws are being passed to help ensure that such AI systems satisfy fairness constraints across different subgroups. A common challenge, however, is that training models to provably guarantee specific behaviors is mathematically difficult, and many existing methods merely encourage certain behaviors rather than guarantee them. To help new AI systems adhere to regulations and satisfy various notions of fairness, we introduce the concept of a "fairness layer": a model component that can be appended to any existing model to ensure that specified fairness constraints are satisfied. Because the layer is differentiable, it can be incorporated into standard end-to-end training pipelines. Experiments show that this fairness layer offers improvements over existing methods for model fairness, as it guarantees constraint satisfaction while retaining predictive model accuracy.