Exploring Nonlinear Pathway in Parameter Space for Machine Unlearning

Machine Unlearning (MU) aims to remove the information of specific training data from a trained model, ensuring compliance with privacy regulations and user requests. While one line of existing MU methods relies on linear parameter updates via task arithmetic, they suffer from weight entanglement. In this work, we propose a novel MU framework called Mode Connectivity Unlearning (MCU) that leverages mode connectivity to find an unlearning pathway in a nonlinear manner. To further enhance performance and efficiency, we introduce a parameter mask strategy that not only improves unlearning effectiveness but also reduces computational overhead. Moreover, we propose an adaptive adjustment strategy for our unlearning penalty coefficient to adaptively balance forgetting quality and predictive performance during training, eliminating the need for empirical hyperparameter tuning. Unlike traditional MU methods that identify only a single unlearning model, MCU uncovers a spectrum of unlearning models along the pathway. Overall, MCU serves as a plug-and-play framework that seamlessly integrates with any existing MU methods, consistently improving unlearning efficacy. Extensive experiments on the image classification task demonstrate that MCU achieves superior performance. The codes are available at https://github.com/TIML-Group/Mode-Connectivity-Unlearning.

When AI models are trained on data that later needs to be removed due to privacy concerns or user requests, we need reliable ways to make them "forget" that information. Existing forgetting methods often update a model's internal settings in a simplistic, straight-line fashion, which can accidentally damage unrelated knowledge the model should keep. We propose a new approach that finds a smarter, curved path through the model's internal space to achieve forgetting more cleanly. Along the way, we automatically identify which parts of the model actually need to be modified, reducing unnecessary computation. We also introduce a self-adjusting mechanism that balances how much the model forgets versus how well it still performs on other tasks without requiring tedious manual tuning. Unlike prior methods that produce just one "forgotten" model, our approach discovers a whole range of valid solutions. Importantly, our framework can be combined with any existing forgetting method to boost its performance.