From Drift to Coherence: Stabilizing Beliefs in LLMs

Large language models (LLMs) are often hypothesized to perform implicit Bayesian inference, yet a key coherence condition—the martingale property of predictive beliefs—has been shown to fail in controlled synthetic in-context learning settings. We revisit this question in a more typical usage regime: generic multiple-choice question answering. Exploiting the discrete answer space, we compute exact predictive distributions and study belief dynamics induced by autoregressive answer resampling. We introduce prompted predictive resampling (PPR), where an LLM generates a sequence of answers to the same question. Empirically, PPR reveals early-stage belief drift, indicating martingale violations. However, after sufficient resampling steps, the belief process self-stabilizes and converges to a coherent predictive distribution. Based on this observation, we further propose (i) a seed-answer prompting strategy to accelerate stabilization, and (ii) a self-consistency loss that amortizes early-stage drift into the model via fine-tuning. Experiments on multiple-choice QA benchmarks show that our methods substantially reduce belief drift and improve predictive coherence without sacrificing accuracy.

Large language models are often thought to update their knowledge logically, implying they possess consistent internal beliefs. However, this consistency had never been mathematically verified for general problem-solving tasks. By analyzing how models answer multiple-choice questions, we discovered they suffer from "belief drift"—their internal probabilities fluctuate erratically when repeatedly asked for answer sequences. To quantify this drift, we introduce a novel method called Prompted Predictive Resampling (PPR) based on consistency mathematics known as martingale posterior theory. We observed that while an AI's confidence wanders initially, it naturally self-stabilizes over time. To accelerate this, we utilized "seed answers" for faster stabilization and fine-tuned the model to satisfy logical consistency rules from its very first response. Across standard benchmarks, these stabilized predictive distributions drastically improve uncertainty calibration while maintaining original accuracy. This proves that mathematical coherence can be operationalized as a powerful target to align AI confidence, creating much more reliable and trustworthy systems for real-world deployment.