Timezone: »
Computing non-vacuous PAC-Bayes generalization bounds for Models under Adversarial Corruptions
Waleed Mustafa · Philipp Liznerski · Dennis Wagner · Puyu Wang · Marius Kloft
Event URL: https://openreview.net/forum?id=Dyqkk0fa3c »
PAC-Bayes generalization bounds have been shown to provide non-vacuous performance certificates for several Machine Learning models. However, under adversarial corruptions, these bounds often fail to maintain their non-vacuous nature due to the increased empirical risk. In this work, we address this limitation by deriving and computing the first non-vacuous generalization bounds for models operating under adversarial conditions. Our approach combines the PAC-Bayes and Adversarial Smoothing frameworks to derive generalization bounds for randomly smoothed models. We empirically demonstrate the efficacy of our bounds in providing robust population risk certificates for stochastic Convolution Neural Networks (CNN) operating under $L_2$-bounded adversarial corruptions for both MNIST and CIFAR-10.
PAC-Bayes generalization bounds have been shown to provide non-vacuous performance certificates for several Machine Learning models. However, under adversarial corruptions, these bounds often fail to maintain their non-vacuous nature due to the increased empirical risk. In this work, we address this limitation by deriving and computing the first non-vacuous generalization bounds for models operating under adversarial conditions. Our approach combines the PAC-Bayes and Adversarial Smoothing frameworks to derive generalization bounds for randomly smoothed models. We empirically demonstrate the efficacy of our bounds in providing robust population risk certificates for stochastic Convolution Neural Networks (CNN) operating under $L_2$-bounded adversarial corruptions for both MNIST and CIFAR-10.
Author Information
Waleed Mustafa (TU Kaiserslautern)
Philipp Liznerski (University of Kaiserslautern-Landau)
Dennis Wagner (Universität Kaiserslautern)
Puyu Wang (City University of Hong Kong)
Marius Kloft (TU Kaiserslautern)
More from the Same Authors
-
2023 Poster: Deep Anomaly Detection under Labeling Budget Constraints »
Aodong Li · Chen Qiu · Marius Kloft · Padhraic Smyth · Stephan Mandt · Maja Rudolph -
2023 Poster: Training Normalizing Flows from Dependent Data »
Matthias Kirchler · Christoph Lippert · Marius Kloft -
2022 Poster: Latent Outlier Exposure for Anomaly Detection with Contaminated Data »
Chen Qiu · Aodong Li · Marius Kloft · Maja Rudolph · Stephan Mandt -
2022 Poster: On the Generalization Analysis of Adversarial Learning »
Waleed Mustafa · Yunwen Lei · Marius Kloft -
2022 Spotlight: Latent Outlier Exposure for Anomaly Detection with Contaminated Data »
Chen Qiu · Aodong Li · Marius Kloft · Maja Rudolph · Stephan Mandt -
2022 Spotlight: On the Generalization Analysis of Adversarial Learning »
Waleed Mustafa · Yunwen Lei · Marius Kloft -
2021 Poster: Neural Transformation Learning for Deep Anomaly Detection Beyond Images »
Chen Qiu · Timo Pfrommer · Marius Kloft · Stephan Mandt · Maja Rudolph -
2021 Spotlight: Neural Transformation Learning for Deep Anomaly Detection Beyond Images »
Chen Qiu · Timo Pfrommer · Marius Kloft · Stephan Mandt · Maja Rudolph -
2018 Poster: Deep One-Class Classification »
Lukas Ruff · Nico Görnitz · Lucas Deecke · Shoaib Ahmed Siddiqui · Robert Vandermeulen · Alexander Binder · Emmanuel Müller · Marius Kloft -
2018 Oral: Deep One-Class Classification »
Lukas Ruff · Nico Görnitz · Lucas Deecke · Shoaib Ahmed Siddiqui · Robert Vandermeulen · Alexander Binder · Emmanuel Müller · Marius Kloft