Infrared-Visible Image Fusion Meets Object Detection: Towards Unified Optimization for Multimodal Perception
Open Access
- 4 November 2025
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 17 (21) , 3637
Abstract
Infrared-visible image fusion and object detection are crucial components in remote sensing applications, each offering unique advantages. Recent research has increasingly sought to combine these tasks to enhance object detection performance. However, the integration of these tasks presents several challenges, primarily due to two overlooked issues: (i) existing infrared-visible image fusion methods often fail to adequately focus on fine-grained or dense information, and (ii) while joint optimization methods can improve fusion quality and downstream task performance, their multi-stage training processes often reduce efficiency and limit the network’s global optimization capability. To address these challenges, we propose the UniFusOD method, an efficient end-to-end framework that simultaneously optimizes both infrared-visible image fusion and object detection tasks. The method integrates Fine-Grained Region Attention (FRA) for region-specific attention operations at different granularities, enhancing the model’s ability to capture complex information. Furthermore, UnityGrad is introduced to balance the gradient conflicts between fusion and detection tasks, stabilizing the optimization process. Extensive experiments demonstrate the superiority and robustness of our approach. Not only does UniFusOD achieve excellent results in image fusion, but it also provides significant improvements in object detection performance. The method exhibits remarkable robustness across various tasks, achieving a 0.8 and 1.9 mAP50 improvement over state-of-the-art methods on the DroneVehicle dataset for rotated object detection and the M3FD dataset for horizontal object detection, respectively.Keywords
This publication has 61 references indexed in Scilit:
- Deep Learning-Based Human Detection Using RGB and IR Images from DronesInternational Journal of Aeronautical and Space Sciences, 2023
- CNN-based fire detection method on autonomous ships using composite channels composed of RGB and IR dataInternational Journal of Naval Architecture and Ocean Engineering, 2022
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- Image fusion meets deep learning: A survey and perspectiveInformation Fusion, 2021
- RFN-Nest: An end-to-end residual fusion network for infrared and visible imagesInformation Fusion, 2021
- SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image FusionInternational Journal of Computer Vision, 2021
- A multi-focus image fusion method based on attention mechanism and supervised learningApplied Intelligence, 2021
- Infrared and visible image fusion via detail preserving adversarial learningInformation Fusion, 2020
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- The TNO Multiband Image Data CollectionData in Brief, 2017