On Calibration of Modern Language Models

Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated response is correct, while calibration aligns these estimates with empirical accuracy. Although token probabilities are readily available during inference, their effectiveness across different confidence estimation strategies remains under-explored. In this work, we systematically study token-probability-based confidence estimators and post-hoc calibration for LLMs on mathematical reasoning tasks. We compare single-pass estimators, which reuse token probabilities from the original generation, with multi-pass estimators, which obtain additional confidence signals through verification or stochastic forward passes. While individual token probabilities can be highly saturated and poorly calibrated, we find that more reliable confidence signals emerge from token-level probability patterns aggregated across the full reasoning trajectory. For multi-pass methods, we study self-verification through model re-prompting and introduce an efficient in-situ variant. We further demonstrate that confidence estimates derived from Monte Carlo Dropout samples achieve better calibration by capturing distributional uncertainty. Finally, we show that post-hoc calibration can recover useful signals from poorly calibrated estimators, with calibration data efficiency depending on dataset difficulty.