Workshop: Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning (ITR3)

A unified PAC-Bayesian framework for machine unlearning via information risk minimization

Sharu Jose · Osvaldo Simeone

Keywords: [ Gaussian Processes and Bayesian non-parametrics ]


Machine unlearning refers to mechanisms that can remove the influence of a subset of training data upon request from a trained model without incurring the cost of re-training from scratch. This paper develops a unified PAC-Bayesian framework for machine unlearning that recovers the two recent design principles -- variational unlearning (Nguyen et al., 2020) and forgetting Lagrangian (Golatkar et al., 2020) -- as information risk minimization problems (Zhang, 2006). Accordingly, both criteria can be interpreted as PAC-Bayesian upper bounds on the test loss of the unlearned model that take the form of free energy metrics.

Chat is not available.