Patch-Wise Infrared and Visible Image Fusion Using Spatial Adaptive Weights
Open Access
- 5 October 2021
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 11 (19) , 9255
Abstract
In infrared (IR) and visible image fusion, the significant information is extracted from each source image and integrated into a single image with comprehensive data. We observe that the salient regions in the infrared image contain targets of interests. Therefore, we enforce spatial adaptive weights derived from the infrared images. In this paper, a Generative Adversarial Network (GAN)-based fusion method is proposed for infrared and visible image fusion. Based on the end-to-end network structure with dual discriminators, a patch-wise discrimination is applied to reduce blurry artifact from the previous image-level approaches. A new loss function is also proposed to use constructed weight maps which direct the adversarial training of GAN in a manner such that the informative regions of the infrared images are preserved. Experiments are performed on the two datasets and ablation studies are also conducted. The qualitative and quantitative analysis shows that we achieve competitive results compared to the existing fusion methods.Keywords
Funding Information
- Institute for Information and Communications Technology Promotion (IITP-2020-0-01462)
- National Research Foundation of Korea (No. 2020R1F1A1077110)
This publication has 27 references indexed in Scilit:
- Infrared and visible image fusion methods and applications: A surveyInformation Fusion, 2019
- Least Squares Generative Adversarial Networks2017
- DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs2017
- Airborne Infrared and Visible Image Fusion Combined with Region SegmentationSensors, 2017
- Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks2016
- A new image quality metric for image fusion: The sum of the correlations of differencesAEU - International Journal of Electronics and Communications, 2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domainInfrared Physics & Technology, 2015
- Image Fusion With Guided FilteringIEEE Transactions on Image Processing, 2013
- Pixel- and region-based image fusion with complex waveletsInformation Fusion, 2007