From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion

Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/

Many real-world vision systems use more than one type of image sensor. For example, visible-light cameras capture texture and color, while infrared sensors can highlight warm objects in dark or difficult environments. Multimodal image fusion aims to combine such images into one result that keeps both clear local details and a consistent overall appearance. Most existing fusion methods represent images as two-dimensional feature grids. These grids are good at preserving local structures such as edges and textures, but they make it harder to control global image properties such as brightness, contrast, and overall visual tone. Our work explores a different representation: a compact one-dimensional token space produced by a pretrained image tokenizer. We use this token space to carry global appearance information, while keeping a conventional two-dimensional pathway to preserve local details. We further introduce Selective Token Editing, which changes only a small number of important tokens to adjust the fused image’s global appearance without adding complex losses or changing the main fusion backbone. Experiments show that this design improves both global coherence and local fidelity across several image fusion benchmarks.