Target-Agnostic Calibration under Distribution Shift with Frequency-Aware Gradient Rectification

Real-world model deployments inevitably encounter distribution shifts, rendering the confidence estimates of deep neural networks highly unreliable, posing severe risks in safety-critical applications. Existing methods improve calibration via training-time regularization or post-hoc adjustment, but often rely on access to (or simulation of) target domains, limiting practicality. We propose Frequency-aware Gradient Rectification (FGR), a target-agnostic training framework for robust calibration. From a frequency perspective, FGR applies low-pass filtering to a subset of training images to diminish spurious high-frequency cues and encourage the learning of domain-invariant features. However, the associated information loss can degrade In-Distribution (ID) calibration. To resolve this trade-off, FGR treats ID calibration as a hard constraint and rectifies conflicting parameter updates via geometric projection. This ensures a first-order non-increase in the ID calibration objective without introducing an additional loss-balancing coefficient. Extensive experiments on synthetic, real-world, and semantic shift datasets demonstrate that FGR significantly improves calibration under diverse shifts while preserving ID performance, and it remains compatible with post-hoc calibration methods. Our code is available at https://github.com/YilinZhang107/FGR-Calib.

AI systems often report confidence in their predictions, but this confidence can become unreliable when test data differs from training data, such as under blur, lighting changes, or new collection environments. Our work aims to make these confidence scores more trustworthy without using examples from the future target environment. We train the model with a small portion of simplified images, encouraging it to rely less on fragile visual details and more on robust information. We also add a training safeguard so this robustness improvement does not harm confidence on ordinary images. Across several image benchmarks, our method improves confidence reliability under many distribution shifts while largely preserving prediction accuracy. This helps make AI systems safer in settings where knowing when a model may be wrong is as important as the prediction itself.