Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads

While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ) offers a promising way to mitigate this issue, but most existing methods are computationally intensive and/or require supervision. In this work, we propose Recurrent Attention-based Uncertainty Quantification (RAUQ), an unsupervised and efficient framework for identifying hallucinations. The method leverages an observation about transformer attention behavior: when incorrect information is generated, certain "uncertainty-aware" attention heads tend to reduce their focus on preceding tokens. RAUQ automatically detects these attention heads and combines their activation patterns with token-level confidence measures in a recurrent scheme, producing a sequence-level uncertainty estimate in just a single forward pass. Through experiments on twelve datasets spanning question answering, summarization, and translation across nine different LLMs, we show that RAUQ consistently outperforms state-of-the-art UQ baselines. Importantly, it incurs minimal overhead, requiring less than 1% additional computation. Since it requires neither labeled data nor extensive parameter tuning, RAUQ serves as a lightweight, plug-and-play solution for real-time hallucination detection in white-box LLMs.

Large language models have become highly capable, but they remain prone to factual inaccuracies, often called "hallucinations." Uncertainty quantification methods can help detect this problem, but many existing methods are computationally expensive or require labeled examples. We propose Recurrent Attention-based Uncertainty Quantification (RAUQ), an unsupervised and efficient method for identifying hallucinations. RAUQ is based on an observation about how large language models process text: when incorrect information is generated, certain internal components of the model tend to reduce their attention on preceding text. Our method automatically detects these components and combines their signals with a confidence measure in a recurrent way. This produces a reliability score for the entire output using only a single model call. Across twelve datasets in question answering, summarization, and translation, and across nine different large language models, RAUQ consistently outperforms strong uncertainty quantification methods. It also adds less than 1% extra computation. Because it does not require labeled data or extensive parameter tuning, RAUQ is a lightweight, plug-and-play solution for real-time hallucination detection in language models.