Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility

Large language models (LLMs) can memorize sensitive facts, motivating unlearning methods that remove targeted knowledge without costly retraining. However, unlearning research remains heavily English-centric. We study multilingual unlearning by extending the TOFU benchmark to five languages, and fine-tune, unlearn and query our models with different permutations of languages. We find that unlearning transfer -- the ability of an unlearned model to "forget" facts in languages other than the unlearning language -- is highly variable: e.g., it is strongest between languages sharing scripts and families, and we show that the unlearning language predicts which query languages are most likely to yield the strongest transfer. Layer-wise analysis reveals that unlearning leaves the shared cross-lingual latent space largely intact in early layers, instead operating primarily in later decoding layers. This suggests that unlearning does not truly erase knowledge, but rather induces superficial suppression. Exploiting this structure, a single inference-time steering direction reverses much of this suppression across languages, recovering 50% (Qwen) and 90% (Gemma) of the unlearned knowledge.

Large language models can inadvertently learn sensitive information from their training data. Unlearning aims to remove selected information from a model. However, most existing work studies unlearning mainly in English, even though these models are used in many languages. This paper studies how unlearning behaves in multilingual language models. We test how information removed in one language transfers to other languages by systematically varying the query language and the unlearning language for the same underlying facts. We also test whether supposedly forgotten information can still be recovered across languages. We find that unlearning transfers unevenly across languages: the effect depends on factors such as language family, script, and pretraining coverage. We also find that current unlearning algorithms can be superficial: a simple steering intervention inside the model can recover much of the forgotten information across languages. These results suggest that unlearning in one language may transfer effectively to similar languages, such as English and German, but may not be sufficient for dissimilar language pairs, such as Chinese and German. Steering vectors could also be a useful tool for testing the robustness of unlearning performance.