Exposing Hidden Biases in Text-to-Image Models via Automated Prompt Search

Text-to-image (TTI) diffusion models have achieved remarkable visual quality, yet they have been repeatedly shown to exhibit social biases across sensitive attributes such as gender, race and age. To mitigate these biases, existing approaches frequently depend on curated prompt datasets - either manually constructed or generated with large language models (LLMs) - as part of their training and/or evaluation procedures. Beside the curation cost, this also risks overlooking unanticipated, less obvious prompts that trigger biased generation, even in models that have undergone debiasing. In this work, we introduce Bias-Guided Prompt Search (BGPS), a framework that automatically generates prompts that aim to maximize the presence of biases in the resulting images. BGPS comprises two components: (1) an LLM instructed to produce attribute-neutral prompts and (2) attribute classifiers acting on the TTI’s internal representations that steer the decoding process of the LLM toward regions of the prompt space that amplify the image attributes of interest. We conduct extensive experiments on Stable Diffusion 1.5 and a state-of-the-art debiased model and discover an array of subtle and previously undocumented biases that severely deteriorate fairness metrics. Crucially, the discovered prompts are interpretable, i.e they may be entered by a typical user, quantitatively improving the perplexity metric compared to a prominent hard prompt optimization counterpart. Our findings uncover TTI vulnerabilities, while BGPS expands the bias search space and can act as a new evaluation tool for bias mitigation.

AI text-to-image generators have been shown to exhibit social biases, like gender or race bias, numerous times by researchers and investigative journalists. In response, new methods have been proposed to "debias" image generation models, either via retraining or by internal adjustements, to give more balanced results. Also, detecting biases in models relies upon manually curated or LLM-generated benchmark datasets of prompts, that are limited in scope and are not similar to real user prompts. Our method, Bias-Guided Prompt search, proposes pairing an LLM with smaller "detector" networks that can peek inside the image generator and predict, before any image is even drawn, the resulting image's likely gender, race, age or any other potentially biased attribute. This way we are able to guide LLM generation towards discovering prompts that contain biases specific to each image generator. We discover that even models that have been debiased contain many residual biases. We further discover biases in many text-to-image models and study how linguistic modifiers influence bias in the models (e.g. "thoughtful" is male-associated while "friendly" is female-associated for Stable Diffusion 1.5).