Stable Localized Conformal Prediction via Transduction

Existing evaluations of conformal prediction, such as prediction efficiency and test-conditional coverage, are defined in expectation over the calibration data. In practice, when only one calibration set of limited size is available, prediction sets often exhibit high variability in size, especially for methods with localization. We formalize this concern as set stability, defined as the variance of the conditional expectation of the set size given the calibration data. To improve stability without requiring additional target-task labels, we propose Stable Conformal Prediction (StCP), a transfer learning approach that utilizes labeled source-task data and unlabeled target data. Theoretically, we characterize the marginal coverage and stability of StCP; empirically, it delivers more stable prediction sets than standard conformal prediction methods, especially for those with localization, when calibration data are limited.

Machine learning systems often provide prediction sets to express uncertainty, but these outputs can become unstable when only a small amount of labeled data is available. In this paper, we formalize this issue through a new notion called set stability, which measures how sensitive prediction sets are to the particular calibration sample and propose Stable Conformal Prediction (StCP) to reduce it. Our approach combines a small labeled dataset with additional unlabeled or related data to produce more reliable and consistent prediction sets while still maintaining strong statistical guarantees. When applied to localized conformal prediction methods, our framework becomes Stable Localized Conformal Prediction (SLCP), which is particularly effective in settings requiring personalized uncertainty estimates. Experiments on real-world datasets show that our method significantly improves stability, especially in data-scarce applications such as medical risk assessment.