Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology. We formalize this bias theoretically, verify it empirically on preference datasets, and show that it plays a central role in mode collapse. Motivated by this analysis, we introduce Verbalized Sampling (VS), a simple, training-free prompting strategy to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., "Generate 5 jokes about coffee and their corresponding probabilities"), which relieves the pressure to produce a single "typical" answer. Experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), social dialogue simulation, synthetic data generation, and open-ended QA, without sacrificing safety and factual accuracy. For instance, in creative writing, VS increases diversity by 1.6-2.1x compared to direct prompting. We further observe an emergent trend that more capable models benefit more from VS. In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.

AI chatbots are often trained to be helpful, safe, and easy to use. But this training can have an unintended side effect: when asked for open-ended ideas, the model may keep returning the same predictable answer. It may know many possible jokes, stories, or ways a person might respond in a conversation, but still choose the most familiar one. Our paper studies why this happens. We find evidence that, when people rate AI answers during training, they often favor responses that feel familiar, fluent, and expected. Over time, this can teach AI systems to “play it safe,” hiding many valid but less typical responses. We introduce Verbalized Sampling, a simple way to bring back this hidden variety. Instead of asking for one answer, we ask the model to generate several possible answers and give their estimated probabilities. For example: “Generate 5 jokes about coffee and their corresponding probabilities.” This small change encourages the model to represent a broader range of plausible answers rather than only the single safest one. We test this method on creative writing, dialogue simulation, automatically creating training examples, and open-ended question answering. In creative writing, it increases diversity by about 1.6–2.1× compared with direct prompting, while keeping quality, factual accuracy, and safety broadly comparable in our experiments. The method does not require retraining or access to model internals, so it can be used with many existing AI systems. More broadly, our work shows that AI diversity may not be destroyed by training models to be helpful and safe; much of it may simply be hidden, and better prompting can help unlock it.