Modality-Decoupled Online Recursive Editing

Online model editing for multimodal large language models (MLLMs) requires assimilating a stream of corrections under tight compute and memory budgets. Yet editors developed for text-only LLMs often degrade on MLLMs: visually dominant activations skew the statistics that shape updates, causing cross-modal conflict, while sequential writes become entangled in a shared edit space and amplify long-horizon interference, causing inter-edit interference. To address these, we propose M-ORE, a modality-decoupled online recursive editor for lifelong MLLM adaptation. M-ORE is derived from a unified proximal-projection formulation and admits a closed-form update with a Sherman-Morrison recursion, yielding constant per-edit overhead. It maintains module-wise locality statistics for the text stack and the visual projector to avoid visually dominated update shaping and performs continual updates in a fixed orthogonal low-rank edit subspace via a Sherman-Morrison recursion to mitigate long-horizon interference. Experiments on multiple MLLM backbones and online editing benchmarks show that our M-ORE method consistently improves reliability, generality, and locality over strong baselines, while achieving favorable quality-efficiency scaling. Our code is publicly available at https://github.com/lab-klc/M-ORE.

Multimodal large language models can answer questions about both images and text, but their knowledge can become outdated or incorrect. Retraining these models from scratch is expensive, so a more practical goal is to correct them through small updates when new information arrives. However, making many such updates over time is difficult: image-related information and text-related information can interfere with each other, and later corrections can accidentally weaken or overwrite earlier ones. This paper introduces M-ORE, a method for updating multimodal models continuously and efficiently. Instead of treating all parts of the model in the same way, M-ORE separates the handling of visual and textual information so that updates from one side do not dominate the other. It also places each new correction into a stable, organized update subspace, which reduces conflicts between many edits made over time. Experiments on visual question answering and image captioning tasks show that M-ORE helps models learn new corrections while better preserving their existing abilities. Compared with strong existing editing methods, it maintains better accuracy after long sequences of updates and keeps the extra computation and memory needed for each update nearly constant. This makes M-ORE a promising step toward more reliable and maintainable multimodal AI systems.