Invited Talk #4 by John Chung: Diversified and Explorative Language Models

While large language models (LLMs) are rapidly advancing in text generation capabilities and in following users' instructions, they still have limitations in supporting humans' creative, divergent, and exploratory tasks. Researchers have identified that LLM outputs tend to have limited diversity, and humans tend to produce more homogenized results when assisted by LLMs. In this talk, I introduce a couple of attempts to align LLMs for more diverse outputs and to support people's explorative creative tasks. First, I introduce a post-training approach to diversify LLM outputs by making them learn more from rarer high-quality instances. Second, I present a user simulator that mimics humans' messy intentions during their creative exploration and show how such a simulator can be used to tune LLMs to be better aligned for explorative and creative tasks. Through this talk, I hope to inspire future researchers to target aligning LLMs for creative and divergent tasks.