Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation

Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text–image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this, we propose DC Attenuation for diVersity Enhancement (\textbf{DAVE}), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.

Ask a text-to-image AI for a picture, then ask again — and you will rarely get something new. These models often keep producing nearly identical images for the same prompt, leaving users with little variety to explore. To find the root cause, we looked inside the model as it draws an image step by step. We discovered that very early in the process, the rough “blueprint” of the whole image — technically, the DC component of the model’s internal features — snaps into nearly the same shape regardless of the random starting point. Once this blueprint locks in, later steps have limited ability to recover broad layout-level diversity. We call this phenomenon early lock-in. We introduce DAVE, a simple method that gently softens this dominant early structure during only the first few steps, giving the AI the freedom to grow different starting points into genuinely distinct layouts. DAVE needs no retraining, no change to how images are generated, and almost no extra computation or memory. Yet it produces far more varied images while preserving quality and prompt-faithfulness — making generative AI more useful for everyday creators exploring visual ideas, and for building the diverse, high-quality datasets that future systems depend on.