HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing

Text-to-Image (T2I) models have made significant strides in visual realism and semantic consistency, yet they often perpetuate and amplify societal biases. Existing evaluation methods typically address only single-dimensional biases, lacking perspectives to uncover model biases at social-related deeper semantic levels. We introduce HoloFair, a comprehensive benchmark framework for multidimensional demographic bias analysis. Built upon our large-scale fairness-oriented dataset and the SpaFreq (Spatial-Frequency) attribute classifier, this framework proposes the Multi-attribute, Group-wise Bias Index (MGBI) metric, designed to assess both intrinsic diversity and conditional biases. Beyond evaluation, we further introduce Fair-GRPO, a reinforcement-learning-based debiasing method that alters the distribution of generative models through a designed multi-objective reward function. E.g., experiments on the SD3.5-Medium model demonstrate that Fair-GRPO significantly improves multidimensional fairness while maintaining high image quality. We also analyze potential reward hacking phenomena and provide corresponding mitigation strategies.

AI image generators can produce remarkably realistic photos from text descriptions. However, when asked to create "a photo of a person," these systems often overwhelmingly generate young white faces — and this skew becomes even more pronounced with prompts like "a professional person." We built a evaluating tool called HoloFair that checks whether image generators treat different genders, ages, and races fairly, not just for simple prompts but also for ones carrying social meaning. We tested eight widely used models and found that some appearing fair on the surface actually hide significant biases triggered by certain words. We then developed a training method called Fair-GRPO that teaches these models to generate more balanced results using a reward-based learning approach. It improved fairness by up to 30% without making the images look worse, and it works across different types of image generators.