Difference-Aware Decision Learning for Multimodal Image Fusion

Multimodal image fusion aims to integrate complementary information from different modalities, but cross-modal discrepancies and local conflicts often make modality allocation uncertain, causing information loss or artifact propagation. We address this problem by formulating fusion as an observation-conditioned probabilistic decision-learning problem, where local modality contribution is explicitly modeled as a decision variable. Based on this view, we propose a dIfference-aware Decision-lEArning muLtimodal image fusion paradigm (IDEAL). IDEAL uses cross-modal differences as decision triggers and constructs spatial and spectral decision conditions from multi-scale difference attention, power-spectrum energy, complementary spectra, and spectral-entropy reliability. These conditions are mapped to interpretable contribution policies through a symmetric Beta prior, while uncertainty modulation pulls unreliable decisions toward conservative mixing when evidence is insufficient. Extensive experiments on multiple fusion tasks demonstrate stable and competitive performance against state-of-the-art methods. Code is available at: https://github.com/Pon915/IDEAL-main.

Multimodal image fusion integrates useful information from different images into a clearer and more complete result. Infrared images emphasize target contours, while visible images provide rich texture details, but local discrepancies or conflicts may occur between them. To address this, we propose IDEAL, a difference-aware fusion method that follows the principle of “perceive differences first, then decide fusion.” IDEAL identifies whether cross-modal differences indicate complementary information or unreliable noise, and adaptively assigns regional modality contributions using spatial, frequency, and reliability cues. When evidence is insufficient, it adopts a conservative strategy to suppress error propagation. Experiments show that IDEAL better preserves targets and details, reduces artifacts, and produces high-quality fused images.