MEDA: Medical-Oriented Activation Editing for Hallucination Mitigation in Medical Large Vision-Language Model

Medical Large Vision-Language Models (Med-LVLMs) suffer from severe hallucinations, posing critical safety risks in clinical deployment. Editing LVLM activations has shown promise for mitigating hallucination with minimal cost. However, due to the requirements of medical domain expertise, existing methods struggle to capture imaging manifestations and diagnostic principles that are critical for clinical interpretation, thereby limiting their effectiveness. To address these limitations, we propose the first MEDical-oriented Activation Editing (MEDA) method by integrating Query-decisive Manifestation Steering (QMS) and Principle-driven Diagnosis Induction (PDI) to promote Med-LVLM's expertise elicitation. Specifically, QMS retrieves positive query-decisive imaging manifestations as trusted guidance for activation steering, while PDI constructs positive principle-embedded diagnostic prompts to induce expert-like clinical reasoning. Extensive experiments across six benchmarks and six LVLMs demonstrate that MEDA efficiently improves the response factuality with up to a 10.2\% gain on IU-Xray, while exhibiting strong generalization and few-shot robustness.

Artificial intelligence (AI) models that can "see" medical images and "talk" about them have the potential to transform healthcare. However, these models often suffer from "hallucinations"—they confidently report medical findings that aren't actually there. In a hospital setting, such errors are dangerous. Current ways to fix these mistakes often fail because they don't understand the complex logic doctors use. To solve this, we developed MEDA, a new method that acts like a specialized "expert coach" for medical AI. MEDA does two things: first, it points the AI’s attention toward specific, critical signs in an image (like a specific shadow on an X-ray); second, it guides the AI to follow established medical principles when reaching a conclusion. Our tests show that MEDA significantly improves the accuracy of AI-generated reports across various medical tasks. By making AI reasoning more disciplined and expert-like, we are moving closer to a future where these tools can safely and reliably assist doctors in saving lives.