Higher-Order Certified Robustness for Regression

Randomized smoothing has emerged as a scalable technique for certifying the adversarial robustness of classifiers. However, its application to regression remains under-explored and faces unique challenges. Existing regression certificates rely on probabilistic acceptance regions and fail to exploit the local geometry of the function. In this work, we present a novel framework for certified robust regression that addresses these limitations. We derive a prediction-centered certificate that guarantees the stability of the smoothed model’s prediction and ensures practical computability at test time. We investigate several alternatives for constructing these certificates by explicitly incorporating means, variances, and gradients. In particular we demonstrate on the MNIST rotation task that utilizing gradient information yields significantly tighter robustness certificates compared to the current state-of-the-art, $\alpha$-smoothing.

Machine learning models can change their predictions after small, carefully chosen input perturbations. This paper studies this problem for regression models, which predict continuous values such as rotation angles or ages. We develop a method for certifying the robustness of a smoothed regression model, meaning that within a certified input neighborhood, the model’s prediction is guaranteed not to change by more than a chosen tolerance. Our method builds on randomized smoothing, which averages predictions over noisy versions of the input, and improves existing regression certificates by using variance and gradient information about the smoothed model. Experiments on synthetic data, rotated digit images, and face age estimation show that this approach gives tighter certified robustness guarantees than prior smoothing-based methods.