Reverse-Engineering Model Editing on Language Models

Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive information. Locate-then-edit methods, as a mainstream paradigm of model editing, offer a promising solution by modifying model parameters without retraining. However, in this work, we reveal a critical vulnerability of this paradigm: the parameter updates inadvertently serve as a side channel, enabling attackers to recover the edited data. We propose a two-stage reverse-engineering attack named KSTER (KeySpaceReconsTruction-then-EntropyReduction) that leverages the low-rank structure of these updates. First, we theoretically show that the row space of the update matrix encodes a "fingerprint" of the edited subjects, enabling accurate subject recovery via spectral analysis. Second, we introduce an entropy-based prompt recovery attack that reconstructs the semantic context of the edit. Extensive experiments on multiple LLMs demonstrate that our attacks can recover edited data with high success rates. Furthermore, we propose subspace camouflage, a defense strategy that obfuscates the update fingerprint with semantic decoys. This approach effectively mitigates reconstruction risks without compromising editing utility. Our code is available at https://github.com/reanatom/EditingAttack.

Large language models are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive information. Current locate-then-edit techniques offer a promising way to erase these memories without retraining. However, we reveal a critical vulnerability of this paradigm: the editing process leaves traces that can leak the data to be edited. We propose a reverse-engineering method (KSTER) that analyzes parameter updates to first pinpoint the target subjects and then reconstruct the context around them. Experiments on multiple models show that KSTER can accurately recover edited information. To fix this, we introduce a camouflage defense strategy that obfuscates the update with semantic decoys. This approach effectively mitigates reconstruction risks without compromising editing utility.