CHITNet: A Complementary to Harmonious Information Transfer Network for Infrared and Visible Image Fusion
Authors:
Preprint
Abstract
Current infrared and visible image fusion (IVIF) methods go to great lengths to excavate complementary features and design complex fusion strategies, which is extremely challenging. To this end, we rethink the IVIF outside the box, proposing a complementary to harmonious information transfer network (CHITNet). It reasonably transfers complementary information into harmonious one, which integrates both the shared and complementary features from two modalities. Specifically, to skillfully sidestep aggregating complementary information in IVIF, we design a mutual information transfer (MIT) module to mutually represent features from two modalities, roughly transferring complementary information into harmonious one. Then, a harmonious information acquisition supervised by source image (HIASSI) module is devised to further ensure the complementary to harmonious information transfer after MIT. Meanwhile, we also propose a structure information preservation (SIP) module to guarantee that the edge structure information of the source images can be transferred to the fusion results. Moreover, a mutual promotion training paradigm (MPTP) with interaction loss is adopted to facilitate better collaboration among MIT, HIASSI and SIP. In this way, the proposed method is able to generate fused images with higher qualities. Extensive experiments demonstrate that CHITNet is superior to state-of-the-art algorithms in terms of visual quality and quantitative evaluations.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- Multi-exposure image fusion via deep perceptual enhancementInformation Fusion, 2022
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- Image fusion meets deep learning: A survey and perspectiveInformation Fusion, 2021
- A unified framework for damaged image fusion and completion based on low-rank and sparse decompositionSignal Processing: Image Communication, 2021
- RFN-Nest: An end-to-end residual fusion network for infrared and visible imagesInformation Fusion, 2021
- Infrared and visible image fusion based on target-enhanced multiscale transform decompositionInformation Sciences, 2020
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- A new image quality metric for image fusion: The sum of the correlations of differencesAEU - International Journal of Electronics and Communications, 2015
- Background-subtraction using contour-based fusion of thermal and visible imageryComputer Vision and Image Understanding, 2007
- A human perception inspired quality metric for image fusion based on regional informationInformation Fusion, 2007