Timezone: »

 
De-stereotyping Text-to-image Models through Prompt Tuning
Eunji Kim · Siwon Kim · Chaehun Shin · Sungroh Yoon

@
in Challenges in Deployable Generative AI »
Event URL: https://openreview.net/forum?id=yNyywJln2R »

Recent text-to-image (TTI) generation models have been reported to generate images demographically stereotyped in various sensitive attributes such as gender or race. This may seriously harm the fairness of the generative model to be deployed. We propose a novel and efficient framework to de-stereotype the existing TTI model through soft prompt tuning. Utilizing a newly designed de-stereotyping loss, we train a small number of parameters consisting of the soft prompt. We demonstrate that our framework effectively balances the generated images with respect to sensitive attributes, which can also generalize to unseen text prompts.

Keywords: Fairness · debias · text-to-image · De-stereotype

Author Information

Eunji Kim (Seoul National University)
Siwon Kim (Seoul National University)
Chaehun Shin (Seoul National University)
Sungroh Yoon (Seoul National University)

More from the Same Authors