De-attribute to Forget for LLM Unlearning

The rapid development of large language models (LLMs) has raised concerns regarding the inclusion of private or inappropriate data during training, which has led to growing interest in LLM unlearning. Many existing LLM unlearning approaches rely on prediction loss-based optimizations, such as maximizing the loss on the forget set. However, these methods often face issues such as over-forgetting and poor model utility. In this work, we address these issues by introducing a novel perspective that shifts the unlearning optimization target to reducing data attribution instead. We propose the first LLM unlearning framework based on data attribution rewards called DareU that employs reinforcement learning to update the LLM and reduce the attribution score of generated responses (i.e., de-attribute) to the forget data owners. Experimental results using an LLM classifier as an efficient approximation of attribution demonstrate that DareU outperforms existing baseline approaches, achieving effective unlearning while balancing forget quality and model utility.

Large language models learn from massive amounts of text, which may include private or copyrighted information that raises data concerns. While current works try to remove the influence of these data from the model by directly updating the model, they often cause the model to output pure gibberish. To address this problem, we propose DareU, which achieves the removal by forcing the model to make its answers untraceable to the data to be removed. Our experiment results demonstrate that DareU outperforms existing baseline approaches, achieving effective removal while balancing preserving the original model's utility well.