A Game-Theoretic Framework for Measuring and Explaining Metric Compatibility in Fair Machine Learning

Machine learning fairness research documents trade-offs but lacks quantitative frameworks to measure intrinsic metric compatibility without requiring causal graphs. We introduce a game-theoretic framework that decomposes metrics into interaction vectors, enabling compatibility measurement between metrics via cosine similarity and mechanistic attribution to attribute coalitions. Through analysis of 6 datasets, 7 models, and 6 debiasing methods, we reveal that fairness and utility are often structurally orthogonal (median compatibility $\approx 0$) rather than diametrically opposed, with conflicts driven by sparse, low-order interactions. We further show that debiasing improves fairness by compressing the compatibility space—reducing compatibility of both synergistic and conflicting relationships—rather than eliminating conflicts, providing a mechanistic basis for understanding metric alignment.

Machine learning is now used to decide who gets a loan, a job, or even a prison sentence. These systems must be both accurate and fair, yet we often do not know whether improving fairness inevitably hurts accuracy, or whether the two goals simply rely on different information. We developed a method that reveals how well different fairness and accuracy scores align with one another. It works by breaking each score into the specific pieces of information it depends on — such as a person's age, income, or gender — and then comparing these ingredient lists across scores. Across many real-world datasets and models, we find that fairness and accuracy are usually not direct enemies. Instead, they often act like independent measuring tapes, tracking different aspects of a model's behavior. When conflicts do arise, they typically come from a few simple patterns in the data, such as imbalances between groups or the influence of particular attributes. This offers practitioners a practical way to diagnose why trade-offs occur and to make more informed choices about which fairness goals to pursue.