Fair Dataset Distillation via Cross-Group Barycenter Alignment

Dataset Distillation aims to compress a large dataset into a small synthetic one while maintaining predictive performance. We show that as different demographic groups exhibit distinct predictive patterns, the distillation process struggles to simultaneously preserve informative signals for all subgroups, regardless of whether group sizes are mildly or severely imbalanced. Consequently, models trained on distilled data can experience substantial performance drops for certain subgroups, leading to fairness gaps. Crucially, these gaps do not disappear by merely correcting group imbalance, since they stem from fundamental mismatches in subgroup predictive patterns rather than from sample-size disparities alone. We therefore formally analyze the interaction between these two sources of bias and cast the solution as identifying a group-imbalance-agnostic barycenter of the predictive information that induces similar representations across all subgroups. By distilling toward this shared aggregate representation, we show that group fairness concerns can be reduced. Our approach is compatible with existing distillation methods, and empirical results show that it substantially reduces bias introduced by dataset distillation. Code is available at https://github.com/mhmoslemi/COBRA.

Modern AI training often needs huge datasets, which is expensive in storage and compute. Dataset distillation solves this by compressing a large dataset into a tiny synthetic one that trains models just as well. However, we discovered that this compression can quietly amplify unfairness: models trained on distilled data can perform much worse for minority demographic groups than models trained on the original data. We traced the problem to how distillation builds its target. Standard methods average across all samples, which lets majority groups dominate and pushes minority patterns out of the compressed dataset. We show that this bias is not just about group sizes; it also depends on how different demographic groups are positioned in the model's internal representation space, and we formalize this through an upper bound on the resulting fairness gap. Our method, COBRA, computes a balanced center point that stays equally close to every demographic group, then distills the synthetic dataset toward that fair target instead. Across seven benchmarks and four distillation methods, COBRA reduces fairness gaps substantially while preserving or improving accuracy. As compressed datasets become common in sensitive areas like healthcare and finance, COBRA helps ensure these efficiency gains do not come at the cost of fairness for underrepresented groups.