A Certified Unlearning Approach without Access to Source Data

With the growing adoption of data privacy regulations, the ability to erase private or copyrighted information from trained models has become a crucial requirement. Traditional unlearning methods often assume access to the complete training dataset, which is unrealistic in scenarios where the source data is no longer available. To address this challenge, we propose a certified unlearning framework that enables effective data removal without access to the original training data samples. Our approach utilizes a surrogate dataset that approximates the statistical properties of the source data, allowing for controlled noise scaling based on the statistical distance between the two. While our theoretical guarantees assume knowledge of the exact statistical distance, practical implementations typically approximate this distance, resulting in potentially weaker but still meaningful privacy guarantees. This ensures strong guarantees on the model's behavior post-unlearning while maintaining its overall utility. We establish theoretical bounds, introduce practical noise calibration techniques, and validate our method through extensive experiments on both synthetic and real-world datasets. The results demonstrate the effectiveness and reliability of our approach in privacy-sensitive settings.

Machine learning models often learn from sensitive data, and new privacy laws require removing specific data from these models. However, traditional removal methods assume the original training data is still accessible, which is frequently unrealistic in practice due to storage limitations, privacy concerns, or regulatory requirements. To solve this, we propose a method which removes data points without needing access to the original training data. Our method uses a surrogate dataset—a substitute that resembles the original data—to safely guide the unlearning process. Specifically, we calculate how different the surrogate data is from the original data and adjust the removal process accordingly, introducing controlled noise to ensure the model behaves as if it never saw the removed data.This is the first approach that provides theoretical privacy guarantees even without original data access. It helps organizations efficiently comply with privacy regulations, even in complex scenarios like outsourced training or restricted data retention. By demonstrating its effectiveness through extensive experiments, including real-world scenarios, our method establishes itself as practical and reliable. This makes machine learning safer and more trustworthy, significantly broadening its potential use in sensitive applications.