Infrared and visible image fusion based on multi-level detail enhancement and generative adversarial network
Authors:
- Xiangrui TianXTXiangrui Tian
- Xiaohan XianyuXXXiaohan Xianyu
- Zhimin LiZLZhimin Li
- Tong XuTXTong Xu
Open Access
- 31 December 2024
- journal article
- Published by OAE Publishing Inc. in Intelligence & Robotics
- Vol. 4 (4) , 524-43
Abstract
Infrared and visible image fusion technology has a wide range of applications in many fields such as target detection and tracking. Existing image fusion methods often overlook the scale hierarchical structure information of features, with local and global features not being closely interconnected. Typically, improvements focus on the network structure and loss functions, while the intimate connection between the quality of the source images and the feature extraction network is often neglected. The aforementioned issues lead to artifacts and blurring of fused images; besides, the detailed edge information can not be well reflected. Therefore, a method of infrared and visible image fusion based on a generative adversarial network (GAN) with multi-level detail enhancement is proposed in this paper. Firstly, the edge information of the input source image is enriched by the multi-level detail enhancement method, which improves the image quality and makes it more conducive to the learning of feature extraction network. Secondly, the residual-dense and multi-scale modules are designed in the generator and the connection between local and global features is established to ensure the transmissibility and coherence of the feature information. Finally, by designing the loss function and dual discriminator constraints to constrain the fusion image, more structure and detail information are added in continuous confrontation. The experimental results show that the fused image contains more detailed texture information and prominent thermal radiation targets. It also outperforms other fusion methods in terms of average gradient (AG), spatial frequency (SF) and edge intensity (EI) metrics, with values surpassing the sub-optimal metrics of 65.41%, 65.09% and 55.22%, respectively.Keywords
This publication has 26 references indexed in Scilit:
- An Infrared and Visible Image Fusion Method Based on Improved GAN with Dropout Layer2024
- Unified gradient- and intensity-discriminator generative adversarial network for image fusionInformation Fusion, 2022
- Image fusion based on generative adversarial network consistent with perceptionInformation Fusion, 2021
- GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image FusionIEEE Transactions on Instrumentation and Measurement, 2020
- NestFuse: An Infrared and Visible Image Fusion Architecture Based on Nest Connection and Spatial/Channel Attention ModelsIEEE Transactions on Instrumentation and Measurement, 2020
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- Image fusion based on object region detection and Non-Subsampled Contourlet TransformComputers and Electrical Engineering, 2017
- Multi-focus image fusion with a deep convolutional neural networkInformation Fusion, 2017
- SSD: Single Shot MultiBox Detector2016
- Dark image enhancement based onpairwise target contrast and multi-scale detail boosting2015