PACCDU: Pyramid Attention Cross-Convolutional Dual UNet for Infrared and Visible Image Fusion
- 28 October 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-16
Abstract
Infrared and visible image fusion is an important branch in the field of information fusion, which aims to integrate effective information from different sensors and enhance the integrity of image information. Therefore, how to fully extract and retain the structural information and texture details in the source images is a pressing problem at present. In this article, a dual-branch UNet fusion network based on pyramidal attention and cross-convolution (PACCDU) is proposed to obtain fused images with high contrast, rich information, and clear contours. The network encoder uses cross-encoding blocks and pyramidal attention blocks to extract contextual features and cross-scale correlation features in different directions. The fusion block uses parallel spatial attention and channel attention to fuse feature information at different scales. The decoding stage uses large kernel convolution blocks and pyramidal attention to reconstruct the fused features. Ablation experiments and comparison experiments are conducted on the public datasets TNO, RoadScene, and NIR, and the results show that the algorithm in this article is superior in both subjective visual and objective evaluation.Keywords
Funding Information
- National Natural Science Foundation of China (61772319, 62002200, 62202268, 62272281)
- Shandong Provincial Science and Technology Support Program of Youth Innovation Team in Colleges (2021KJ069, 2019KJN042)
- Yantai Science and Technology Innovation Development Plan (2022JCYJ031)
This publication has 41 references indexed in Scilit:
- Hierarchical multi-modal fusion FCN with attention model for RGB-D trackingInformation Fusion, 2019
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- DenseFuse: A Fusion Approach to Infrared and Visible ImagesIEEE Transactions on Image Processing, 2018
- Infrared and Visible Image Fusion using a Deep Learning Framework2018
- Squeeze-and-Excitation Networks2018
- DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs2017
- Infrared and visible image fusion via gradient transfer and total variation minimizationInformation Fusion, 2016
- Rethinking the Inception Architecture for Computer Vision2016
- A new image fusion performance metric based on visual information fidelityInformation Fusion, 2013
- Multi-spectral SIFT for scene category recognition2011