Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence

Parameter-based knowledge editing updates the internal knowledge of large language models (LLMs) via localized weight modifications and has attracted significant attention. However, most existing methods overlook fundamental theoretical limitations and are rarely evaluated under realistic, practice-oriented settings. In this paper, we first present a theoretical analysis based on the dimensional Collapse Hypothesis, explaining how localized parameter edits can propagate along fragile directions in the representation space, inducing global interference and ultimately causing reasoning collapse. Building on this insight, we conduct a comprehensive empirical evaluation by systematically varying knowledge complexity, number of edits, evaluation dimensions, and baseline methods. Our results show that parameter-based editing methods consistently damage core LLM capabilities. In contrast, a simple retrieval-based baseline achieves consistently stronger performance than all parameter-editing methods across all evaluated conditions. These findings highlight that preserving the fundamental capabilities of LLMs after knowledge editing should be a central concern for future research.

Large language models sometimes need to update specific information without being retrained from scratch. A popular approach is to directly change a small part of the model’s internal parameters, but it remains unclear whether this can be done safely without harming the model more broadly. In this paper, we show theoretically that even small, localized parameter changes can spread through the model in unexpected ways, interfering with other stored knowledge and reasoning abilities. We also argue that existing evaluations of knowledge editing are often too narrow and do not fully reflect how these methods would be used in practice. To address this, we carry out a broader and more realistic evaluation, including more complex knowledge, repeated updates, stronger tests of the model’s core abilities, and comparisons with methods that rely on retrieving external information instead of rewriting the model itself. We find that parameter-based editing methods consistently damage important capabilities of large language models. By contrast, a simple retrieval-based baseline performs better across all settings we tested. Our results suggest that future work on knowledge updating should focus not only on whether an edit succeeds, but also on whether the model remains reliable afterward.