Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic Space

Infrared and visible image fusion aims to integrate complementary modalities, while existing Euclidean methods impose rigid distance metrics that distort multi-modal interactions and parent-to-child semantic hierarchies. To overcome these limitations, we introduce a text-driven fusion framework empowered by hyperbolic manifold learning. During training, BLIP-extracted text prompts serve as topological anchors within the hyperbolic space, guiding vision-attribute alignment through hyperbolic embeddings that naturally accommodate varying semantic granularities. By exploiting the exponential volume growth dictated by the Poincaré ball's negative curvature, this approach seamlessly embeds hierarchical trees to encode coarse-to-fine semantics without metric saturation, while the vast peripheral space prevents texture distortion during cross-modal fusion. At inference, the fusion process autonomously adapts to input content using the learned text-attribute priors, completely eliminating the need for textual input. Experimental results show our method outperforms state-of-the-art approaches on benchmark datasets, with code available at https://github.com/Shaoyun2023/TEDFusion.

Combining infrared and visible camera images helps computers see in difficult environments, but current fusion methods struggle to blend these modalities cleanly. Traditional methods rely on flat geometry that fails to capture hierarchical relationships, often distorting fine textures when matching broad concepts with small details. To solve this, we introduce a framework that uses curved geometry. This mathematical space expands naturally like the branches of a tree, allowing the network to organize details from large objects down to fine surface textures without distortion. During training, we use descriptive text to guide this organization, though the model operates entirely on its own during final use. Our method outperforms existing technology on standard benchmarks, and the code is available at https://github.com/Shaoyun2023/TEDFusion.