Robust Bayes-Assisted Conformal Prediction

Bayes--assisted conformal prediction combines the strengths of Bayesian modelling with exact, distribution--free frequentist coverage guarantees. While validity holds even under model misspecification, the size of the prediction sets can degrade significantly when the prior is poorly aligned with the observed data. We address this limitation by introducing \textbf{RoBAS}: a novel Bayes--assisted nonconformity score which is motivated by a hierarchical Bayesian working model with heavy--tailed priors, and which we implement in practice via a computationally tractable empirical Bayes instantiation. Our proposed method is adaptive to the quality of the available working information in the prior. When reliable prior information is available and can be effectively encoded, we achieve set sizes lower than that of other sets with the same coverage. On the other hand, when such information is weak or inaccurate, our nonconformity scores revert to the Distance--To--Average score, a robust baseline that is well--suited to settings where accurate prior information is not available. We evaluate our method on tabular and image regression tasks in the setting where there exists distribution shift between the training and calibration/test data. We find that our approach is competitive with widely used nonconformity scores in the absence of distribution shift, while providing significant gains in the more challenging setting of distribution shift.