Unlearning’s Blind Spots: Over‑Unlearning and Prototypical Relearning Attack

Machine unlearning (MU) aims to expunge a designated forget set from a trained model without costly retraining, yet the existing techniques overlook two critical blind spots: "over‑unlearning" that deteriorates retained data near the forget set, and post‑hoc "relearning" attacks that aim to resurrect the forgotten knowledge. Focusing on class-level unlearning, we first derive an over-unlearning metric, $\operatorname{OU}@\varepsilon$, which quantifies collateral damage in regions proximal to the forget set, where over-unlearning mainly occurs. Next, we expose an unforeseen relearning threat on MU, i.e., the Prototypical Relearning Attack, which exploits the per-class prototype of the forget class with just a few samples, and easily restores the pre-unlearning performance. To counter both blind spots in class-level unlearning, we introduce $\texttt{Spotter}$, a plug‑and‑play objective that combines (i) a masked knowledge‑distillation penalty on the nearby region of forget classes to suppress $\operatorname{OU}@\varepsilon$, and (ii) an intra‑class dispersion loss that scatters forget-class embeddings, neutralizing Prototypical Relearning Attacks. $\texttt{Spotter}$ achieves state-of-the-art results across CIFAR, TinyImageNet, and CASIA-WebFace datasets, offering a practical remedy to unlearning’s blind spots.

When an AI model learns from data that later needs to be removed, for example, a person's face from a facial recognition system, retraining the model from scratch is extremely costly. "Machine unlearning" offers a faster alternative by selectively erasing specific knowledge from a trained model. However, existing unlearning methods have two hidden weaknesses that have gone unnoticed. First, they often erase too much, accidentally degrading the model's performance on data that was supposed to stay intact, especially data that is similar to what was erased. Second, even after unlearning, the model's internal representations subtly retain enough structure that an attacker can recover the erased information using just a handful of images, which we call a Prototypical Relearning Attack. We addressed both problems with a new unlearning framework called Spotter. It uses two targeted techniques: one that protects data near the erased class from collateral damage, and another that scrambles the internal feature structure of the erased class so it can no longer be reconstructed by an attacker. Our results show that Spotter reliably fixes both blind spots across a range of datasets and model types, making machine unlearning more trustworthy for real-world privacy applications.