Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks

The deployment of quantized neural networks on edge devices, combined with privacy regulations like GDPR, creates an urgent need for machine unlearning in quantized models. However, existing methods face critical challenges: they induce forgetting by training models to memorize incorrect labels, conflating forgetting with misremembering, and employ scalar gradient reweighting that cannot resolve directional conflicts between gradients. We propose $\textbf{OEU}$, a novel Orthogonal Entropy Unlearning framework with two key innovations: 1) Entropy-guided unlearning provides an unbiased forgetting direction by maximizing prediction uncertainty on forgotten data, avoiding confident misprediction toward any specific class, and 2) Gradient orthogonal projection eliminates interference by projecting forgetting gradients onto the orthogonal complement of retain gradients, providing theoretical guarantees for utility preservation under first-order approximation. Extensive experiments demonstrate that OEU outperforms existing methods in both forgetting effectiveness and retain accuracy.

Privacy laws like GDPR give people the right to have their data removed from AI systems. This is especially challenging for compressed AI models designed for resource-constrained devices: existing removal methods make the model memorize wrong answers instead of truly forgetting, and can harm its overall performance. We propose Orthogonal Entropy Unlearning (OEU), which guides the model toward genuine uncertainty on forgotten data — as if it had never seen it — rather than teaching it incorrect answers. Our method also mathematically prevents the forgetting process from degrading performance on retained data. Experiments show OEU more closely matches a fully retrained model while preserving accuracy, enabling practical privacy compliance for AI on resource-constrained devices.