Gradients with Respect to Semantics Preserving Embeddings Tell the Uncertainty of Large Language Model

Uncertainty quantification (UQ) is an important technique for ensuring the trustworthiness of LLMs, given their tendency to hallucinate. Existing state-of-the-art UQ approaches for free-form generation rely heavily on sampling, which incurs high computational cost and variance. In this work, we propose the first gradient-based UQ method for free-form generation, SemGrad, which is sampling-free and computationally efficient. Unlike prior gradient-based methods developed for classification tasks that operates in parameter space, we propose to consider gradients in semantic space. Our method builds on the key intuition that a confident LLM should maintain stable output distributions under semantically equivalent input perturbations. We interpret the stability as the gradients in semantic space and introduce a Semantic Preservation Score (SPS) to identify embeddings that best capture semantics, with respect to which gradients are computed. We further propose HybridGrad, which combines the strengths of SemGrad and parameter gradients. Experiments demonstrate that both of our methods provide efficient and effective uncertainty estimates, achieving superior performance than state-of-the-art methods, particularly in settings with multiple valid responses.

Large language models can produce convincing but incorrect answers, and users often do not know when to trust them. Existing methods for estimating a model’s uncertainty often ask the model to generate many alternative answers and compare them, which is slow and can vary from run to run. We propose a new way to estimate uncertainty by looking inside an open-source language model during generation. The key idea is that when a model truly understands a question, its answer probabilities should not change much if the question is rephrased with the same meaning. Our method, SemGrad, measures how sensitive the model’s output is to small changes in internal representations that preserve the input’s meaning. We also introduce a score for finding which internal representations best capture meaning, and combine semantic gradients with standard parameter gradients in a method called HybridGrad. Across question-answering benchmarks, these methods more accurately predicted whether the model’s answer was correct, especially for questions with multiple valid answers. This work makes uncertainty estimation for language models more efficient and helps build systems that can better signal when their answers may be unreliable.