From Individual Calibration to Reliable Classifiers: ALD Parameterization with mPAIC Guarantees

Modern neural classifiers can achieve remarkable predictive performance, yet often suffer from miscalibration. In this paper, we introduce a unified calibration framework applicable to arbitrary distribution-based classifiers. The proposed calibration objective guarantees a monotone Probably Approximately Individually Calibrated (mPAIC) predictor, which theoretically implies the properties of a Probably Approximately Calibrated Classifier (PACC) with explicit error bounds. To enable stable and effective optimization, we further devise a Decoupled Dual-Stream Optimization (DDSO) strategy with gradient detachment to reconcile discriminative representation learning and continuous calibration. Notably, our framework bridges calibration paradigms, supporting flexible deployment either as an end-to-end pre-calibration objective or as a lightweight post-calibration adapter. Extensive experiments across nine real-world datasets demonstrate that our approach consistently outperforms strong baselines, achieving superior performance on both accuracy and multi-level calibration.

When an AI system says it is "90% confident," can you trust that number? In high-stakes applications like medical diagnosis, the reliability of confidence scores matters as much as accuracy. Unfortunately, modern neural networks are often overconfident—their stated probabilities do not reflect how often they are actually correct. We trace a key source of this problem to the Softmax function, which implicitly assumes every input carries the same level of uncertainty. We replace it with a more flexible statistical model that adapts its uncertainty estimate to each input, and pair it with a training objective that provably aligns confidence scores with true correctness rates. A carefully designed optimization strategy ensures calibration improvements do not come at the expense of accuracy. Experiments across nine real-world datasets show our approach consistently produces models that are both accurate and reliably calibrated.