MFH-NAS:A Hybrid Neural Architecture Search Framework for Multimodal Fusion Object Detection

Multimodal fusion object detection faces a substantial modality gap at the same backbone stage. This makes predefined stage-aligned fusion insufficient for cross-stage interactions. We propose MFH-NAS, a hybrid neural architecture search framework that automatically discovers fusion architectures to better leverage cross-modal complementarity. MFH-NAS searches both local fusion primitives and stage-level fusion connectivity. It targets fusion operator design and fusion stage selection. It couples differentiable search with evolutionary search. Differentiable search learns architecture parameters for local fusion primitives. Evolutionary search explores global fusion topologies, including stage selection and cross-stage connection patterns. The joint search balances exploitation and exploration and mitigates premature convergence. It yields fusion structures that strengthen cross-stage interactions.We evaluate MFH-NAS on three public benchmarks, LLVIP, RGBT-Tiny, and M3FD. MFH-NAS consistently outperforms handcrafted fusion-stage designs and prior stage-searching NAS baselines, improving mAP@0.5 from 85.3% to 88.2% over strong fixed-stage fusion methods and delivering gains across all benchmarks.The code is available at https://github.com/someboy0/MFH-NAS.

Many computer vision systems need to detect objects in difficult conditions such as darkness, fog, or glare. Visible-light cameras and infrared cameras provide complementary information, but it is not obvious where in a detection model their information should be combined. Existing systems often fuse the two sources at fixed stages, which can miss useful connections between low-level and high-level features. We propose MFH-NAS, a method that automatically searches for a good fusion design. Instead of asking researchers to hand-design every fusion point, it decides both where to combine the two modalities and which fusion operation to use. The method combines a fast search strategy for local choices with a broader search strategy for the overall model structure. On three public visible-infrared object detection benchmarks, MFH-NAS improves detection accuracy compared with hand-designed fusion methods and previous architecture-search baselines. This can help build more reliable perception systems for applications such as autonomous driving, robotics, and safety monitoring, especially in challenging lighting or weather conditions.