Fair Learning with Biased Labels: When Observed Accuracy Is the Wrong Target

In fair machine learning, the goal is to learn fair models from datasets that are inherently biased against certain groups, while most methods evaluate performance on datasets that are also biased, which often results in an undesirable fairness–accuracy trade-off. In this work, we argue that such trade-off partially results from a category error that fails to distinguish observation and construct. To illustrate this distinction, we consider a learning problem under an explicit worldview that models data bias directly: the samples are drawn from an ideal, unbiased distribution in the construct space, but the observed labels are then corrupted by group-dependent noise. Facing the uncertainty of noisy process, we introduce a minimax learning framework to optimize on the potential underlying unbiased distribution, instead of observed accuracy. Our analysis shows that the new learning problem can be described by minimizing a novel corrected risk, which decomposes exactly into the observed risk minus a closed-form correction. This leads to learning algorithms that can be deployed with access only to biased dataset. Experiments on semi-synthetic datasets demonstrate that our methods improve both accuracy and fairness over unconstrained optimization when evaluated on the unbiased distribution. Questioning the presumed inevitability of fairness–accuracy trade-offs, this work offers a novel framework for fair learning under biased labels.