REPO: Detoxifying LLMs via Representation Erasure-based Preference Optimization

Large language models (LLMs) trained on webscale data can produce toxic outputs, raising concerns for safe deployment. Prior defenses based on DPO, NPO, and similar algorithms reduce the likelihood of harmful continuations but not robustly: they are vulnerable to adversarial prompting and relearning attacks, and linear probing reveals that harmful ``directions'' remain present in representations. We propose Representation Erasure-based Preference Optimization (\REPO), which reformulates detoxification as a token-level preference problem, forcing the representations of toxic continuations to converge toward their benign counterparts. Unlike baselines, \REPO induces deep, localized edits to toxicity-encoding neurons while preserving utility, achieving state-of-the-art robustness against relearning attacks and enhanced GCG jailbreaks where existing methods fail.