Front-Loaded Robust Conformal Prediction: Heavy Calibration, Minimal Test-Time Cost

Robust conformal prediction (RCP) extends conformal prediction (CP) to noisy inputs by producing prediction sets with guaranteed coverage, ensuring that the true label is contained in the set with a user-specified probability even under worst-case perturbations. Recent works use randomized smoothing, as it provides robustness for black-box models at larger radii. Currently, there exist two setups for smoothing-based RCP: one requires extensive Monte Carlo sampling at calibration and test time but results in smaller prediction sets; the other setup produces larger prediction sets but uses a single sample at both stages. In deployment, calibration—as a one-time pre-processing step—can accommodate substantially higher computational overhead than inference. Inspired by this observation, we introduce an RCP framework that strikes a balance between the two extremes of this trade-off: we increase the sample rate at calibration time while keeping it either one or very low during test time. This calibration-time sampling opens the possibility of reducing the size of the prediction sets. In production, where the number of test predictions typically far exceeds the size of the calibration set, our Front-Loaded RCP matches the computational complexity of the state of the art while producing considerably smaller prediction sets at larger radii.

AI systems used in high-stakes settings—such as medical diagnosis or autonomous driving—must communicate not just a prediction but a reliable measure of their uncertainty. Conformal prediction as a representation of such uncertainty returns a set of plausible answers guaranteed to contain the correct solution with high probability. However, this guarantee collapses when inputs are slightly corrupted by noise or adversarial manipulation. Robust conformal prediction addresses this issue. Among existing approaches, a fundamental trade-off exists: methods that run the model hundreds of times per input produce small, informative sets but are too slow for real-time use, while methods that use a single model evaluation are fast but return larger, less informative sets. We resolve this tension by observing that the one-time setup phase before deployment—calibration—can absorb high computational cost without affecting response time. Concentrating all heavy computation in calibration while keeping inference to a single model evaluation We additionally address a separate limitation of single-evaluation methods: their outputs are inherently random, meaning the same input can yield unpredictably large sets across runs. By accounting for (low-rate) repeated sampling at inference time we extend our method to produce near-deterministic, consistently sized sets without sacrificing the robustness guarantee. Importantly, both methods require no knowledge of the model's internal architecture, training procedure, or data distribution, making them directly applicable to any existing AI system as a post-hoc wrapper.