Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents

Large-language models (LLMs) and chatbot agents are known to provide wrong outputs at times, and it was recently found that this can never be fully prevented. Hence, uncertainty quantification plays a crucial role, aiming to quantify the level of ambiguity in either one overall number or two numbers for aleatoric and epistemic uncertainty. This position paper argues that this traditional dichotomy of uncertainties is too limited for the open and interactive setup that LLM agents operate in when communicating with a user, and that we need to research avenues that enrich uncertainties in this novel scenario. We review the literature and find that popular definitions of aleatoric and epistemic uncertainties directly contradict each other and lose their meaning in interactive LLM agent settings. Hence, we propose three novel research directions that focus on uncertainties in such human-computer interactions: Underspecification uncertainties, for when users do not provide all information or define the exact task at the first go, interactive learning, to ask follow-up questions and reduce the uncertainty about the current context, and output uncertainties, to utilize the rich language and speech space to express uncertainties as more than mere numbers. We expect that these new ways of dealing with and communicating uncertainties will lead to LLM agent interactions that are more transparent, trustworthy, and intuitive.

When Artificial Intelligence (AI) models generate text, it sometimes includes wrong information. This happens when the model is uncertain, either because it does not know a certain fact or because a question is not clear enough. Previously, there were many debates in the field about what types of uncertainties there are. We show that researchers have come to conflicting definitions of the types, and that the types of uncertainties do not function anymore in chatbots. Instead, we propose to focus more on why an AI model becomes uncertain, how it can resolve its questions by asking the user, and how it should speak about its uncertainties. We believe that research on these directions will lead to more trustworthy AI models in the future.