CHITNet: A Complementary to Harmonious Information Transfer Network for Infrared and Visible Image Fusion
- 9 January 2025
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 74 (00189456) , 1-17
Abstract
Current infrared and visible image fusion (IVIF) methods go to great lengths to excavate complementary features and design complex fusion strategies, which is extremely challenging. To this end, we rethink the IVIF outside the box, proposing a complementary to harmonious information transfer network (CHITNet). It reasonably transfers complementary information into harmonious one, which integrates both the shared and complementary features from two modalities. Specifically, to skillfully sidestep aggregating complementary information in IVIF, we design a mutual information transfer (MIT) module to mutually represent features from two modalities, roughly transferring complementary information into harmonious one. Then, a harmonious information acquisition supervised by source image (HIASSI) module is devised to further ensure the complementary to harmonious information transfer after MIT. Meanwhile, we also propose a structure information preservation (SIP) module to guarantee that the edge structure information of the source images can be transferred to the fusion results. Moreover, a mutual promotion training paradigm with interaction loss is adopted to facilitate better collaboration among MIT, HIASSI and SIP. In this way, the proposed method is able to generate fused images with higher qualities. Extensive experimental results demonstrate the superiority of CHITNet over state-of-the-art algorithms in terms of visual quality and quantitative evaluations. The source code of the proposed method can be available at https://github.com/lhf12278/CHITNet.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62161015)
- Yunnan Fundamental Research Projects (202301AV070004, 202401AS070106)
This publication has 63 references indexed in Scilit:
- A Deep Learning Framework for Infrared and Visible Image Fusion Without Strict RegistrationInternational Journal of Computer Vision, 2023
- Fusion from Decomposition: A Self-Supervised Decomposition Approach for Image Fusion2022
- A unified framework for damaged image fusion and completion based on low-rank and sparse decompositionSignal Processing: Image Communication, 2021
- SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image FusionInternational Journal of Computer Vision, 2021
- The Eighth Visual Object Tracking VOT2020 Challenge Results2020
- The TNO Multiband Image Data CollectionData in Brief, 2017
- A new image quality metric for image fusion: The sum of the correlations of differencesAEU - International Journal of Electronics and Communications, 2015
- Image fusion of visible and thermal images for fruit detectionBiosystems Engineering, 2009
- Background-subtraction using contour-based fusion of thermal and visible imageryComputer Vision and Image Understanding, 2007
- A human perception inspired quality metric for image fusion based on regional informationInformation Fusion, 2007