MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts

Large Language Models (LLMs) hold great promise for healthcare applications, but fast-changing medical knowledge can quickly make their outputs outdated or inaccurate, limiting use in high-stakes settings. Model editing can update LLMs without full retraining, but parameter-based methods often break locality and are risky in medicine, making retrieval-based editing a better fit. However, applying model editing methods to the medical domain has two key challenges: (1) retrieval-based methods suffer from representation overlap within the medical knowledge space that causes inaccurate retrieval and reduces editing accuracy; (2) existing medical editing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world applications. To address these challenges, we construct MedVersa, an expanded benchmark that evaluates single and batch edits across broader medical coverage under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query–key module for precise matching with an attention-based prompt encoder for informative guidance. Experiments across various medical benchmarks show that our MedREK consistently improves key metrics and provides the first validated solution for batch editing in medical LLMs.

Large language models (LLMs) are becoming increasingly useful for tasks such as answering medical questions and supporting clinical decision-making. However, medical knowledge changes quickly as new diseases, treatments, and guidelines appear. This means LLMs can easily become outdated and provide incorrect or unreliable information. One way to solve this problem is to directly update specific pieces of knowledge inside an LLM instead of retraining the entire model, which is extremely expensive. However, existing methods often struggle in medicine because many medical concepts are highly similar to one another. As a result, the model may retrieve the wrong information or accidentally affect unrelated medical knowledge. In addition, most previous work only updates one medical fact at a time, even though real healthcare applications may require many updates simultaneously. To address these challenges, we created MedVersa, a new benchmark that tests how well medical LLMs can safely handle both single and large groups of updates. We also developed MedREK, a new method that helps LLMs better retrieve and apply the correct medical knowledge. Our experiments show that MedREK produces more accurate and reliable updates while preserving unrelated information, making medical LLMs safer and more practical for real-world healthcare use.