Beyond Reactivity: Proactive Adaptive Conformal Inference for Online LLM Factuality

Large Language Models (LLMs) often produce hallucinated outputs, which limit their reliability in high-stakes applications. Conformal prediction can provide guarantees on the correctness and factuality of LLM outputs, but existing approaches rely on the exchangeability assumption, which rarely holds in online settings where user queries and interests change over time. To solve this problem, in this paper, we propose PACE (Proactive Adaptive Conformal InferencE), a novel framework that sequentially updates the time-varying target miscoverage parameter with a dynamic step size to maintain valid coverage under online distribution shifts. PACE is motivated by the theoretical connections between expected miscoverage error and key factors such as distribution shifts and instantaneous parameter error. It integrates two complementary signals: (1) a proactive shift detection to estimate the magnitude of distribution shifts, and (2) a reactive error that scales updates according to the local coverage gap. Extensive experiments on synthetic and real-world datasets demonstrate that PACE consistently outperforms advanced adaptive baselines. It reduces the deviation from the target error rate by up to 60\% in QA tasks and accelerates coverage recovery by over 2.5x during abrupt shifts, ensuring stable factuality guarantees without compromising utility and stability.

Large language models are increasingly used to answer questions and generate factual text, but they can become unreliable when the kinds of questions they receive change over time. A system that was carefully checked on one set of examples may give too many incorrect or unsupported answers when users start asking about new topics or harder cases. We study how to keep such systems reliable in changing environments. Our proposed method PACE monitors both recent mistakes and whether incoming questions differ from the examples the system was originally checked on. When the incoming questions appear to shift, PACE adjusts how cautious the system should be before too many errors build up. In our experiments, PACE kept the system’s error rate closer to the chosen target while avoiding unnecessarily large answer sets during easier periods. This provides a practical way to express uncertainty: the system gives broader answer sets when it has less evidence to be confident, and narrower ones when the input stream is familiar and stable.