Cheap Forgetting: Linear Adapter Interpolation as a Post-Hoc Memorization Mitigation

Foundation models verbatim-memorize portions of their training data, but the established mitigations, such as differentially private training and machine unlearning, are expensive. We measure whether a much cheaper alternative, linear interpolation between a fine-tuned model and its base, can suppress memorization without destroying utility. Using Llama-3.2-1B fine-tuned with LoRA on a corpus containing 100 planted 16-digit canaries, we sweep the interpolation coefficient $\alpha$ from 0 to 1 and find that extraction collapses from 97% at $\alpha=1$ to 0% at $\alpha=0.5$, while held-out language-modeling NLL recovers from 2.39 to 2.20 (matching the base model). Compared to early stopping at matched utility, $\alpha$-merging extracts $\approx 10\times$ less (7% vs 71% at NLL $\approx 2.25$). We position this as an empirical post-hoc mitigation, not a formal privacy guarantee, and discuss its place alongside differential privacy and unlearning. Code and configurations are available at https://anonymous.4open.science/r/Memorizationpaper/.