Non-Parametric Probabilistic Robustness: A Conservative Risk Estimator under Unknown Perturbation Distributions

Deep learning (DL) models, despite their remarkable success, remain vulnerable to small input perturbations that can cause erroneous outputs, motivating the recent proposal of probabilistic robustness (PR) as a complementary alternative to adversarial robustness (AR). However, existing PR formulations assume a fixed and known perturbation distribution, an unrealistic expectation in practice. To address this limitation, we propose non-parametric probabilistic robustness (NPPR), a more practical PR metric that does not rely on any predefined perturbation distribution. Following the non-parametric paradigm in statistical modeling, NPPR learns an optimized perturbation distribution directly from data, enabling conservative PR evaluation under distributional uncertainty. We further develop an NPPR estimator based on a Gaussian Mixture Model (GMM), covering various input-dependent and input-independent perturbation scenarios. Theoretical analyses establish the relationships among AR, PR, and NPPR. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet across ResNet18/50, WideResNet50 and VGG16 validate NPPR as a more practical robustness metric, showing conservative (lower) PR estimates compared to assuming those common perturbation distributions used in state-of-the-arts.

Deep learning models are now widely used in tasks such as image recognition, but they can still make wrong predictions when the input is changed only slightly. Researchers usually study this issue through adversarial robustness, which asks whether a model can resist carefully designed attacks. Probabilistic robustness offers a complementary view by asking how likely a model is to remain correct under random input changes. However, existing methods often assume that these input changes follow a known and fixed pattern, which is rarely true in real applications. In this work, we propose non-parametric probabilistic robustness (NPPR), a new way to evaluate model robustness without requiring a predefined perturbation distribution. Instead, NPPR learns a challenging perturbation distribution directly from data, leading to a more conservative and practical estimate of model reliability. We also develop a Gaussian-mixture-based estimator that can handle different types of input changes. Our theoretical and experimental results show that NPPR provides a useful bridge between adversarial robustness and probabilistic robustness, and gives a more realistic assessment of model vulnerability across several datasets and neural network architectures.