Optimizing Diversity and Quality through Base-Aligned Model Collaboration

Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations, especially in open-ended generation tasks. We propose Base-Aligned Model Collaboration (BACo), an inference-time token-level model collaboration framework that dynamically combines a base LLM with its aligned counterpart to optimize diversity and quality. Using uncertainty and content-based signals, BACo employs routing strategies to determine, at each token, which model to decode from. Prior diversity-promoting methods often improve diversity at the expense of quality or require expensive decoding or post-training. In contrast, BACo achieves both high diversity and quality post hoc within a single pass, while offering strong controllability. We introduce a family of effective routing strategies and evaluate them across three open-ended generation tasks with 13 diversity and quality metrics. BACo consistently surpasses state-of-the-art inference-time baselines. With our best router, BACo achieves a 21.3% joint improvement in diversity and quality, which is further supported by human evaluations. Overall, our results demonstrate that collaboration between base and aligned models provides an effective and controllable mechanism for optimizing the diversity-quality trade-off.

Modern AI chatbots are trained to be polished and helpful, but this comes with a hidden cost. When you ask the same open-ended question many times, say, "suggest a summer vacation destination," a polished model tends to repeat nearly the same response again and again. The earlier, "raw" version of the same model gives far more diverse responses, but they are often low in quality. So you seem stuck choosing between diversity and quality. We built BACo, a method that lets these two versions of a model team up while writing, handing off to each other word by word. The polished model takes over where accuracy and fluency matter, while the raw model steps in where there is room to diverge. Simple rules decide who writes each word, and a single adjustable dial lets users choose how much diversity they want. The result is text that is both diverse and high quality, produced in one pass without retraining the AI or running it many times. This could help with dialogue, brainstorming, writing, and any task where exploring many good options is appreciated.