Multi-scale attention-based lightweight network with dilated convolutions for infrared and visible image fusion
Authors:
Open Access
- 3 August 2023
- journal article
- research article
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 10 (1) , 705-719
Abstract
Infrared and visible image fusion aims to generate synthetic images including salient targets and abundant texture details. However, traditional techniques and recent deep learning-based approaches have faced challenges in preserving prominent structures and fine-grained features. In this study, we propose a lightweight infrared and visible image fusion network utilizing multi-scale attention modules and hybrid dilated convolutional blocks to preserve significant structural features and fine-grained textural details. First, we design a hybrid dilated convolutional block with different dilation rates that enable the extraction of prominent structure features by enlarging the receptive field in the fusion network. Compared with other deep learning methods, our method can obtain more high-level semantic information without piling up a large number of convolutional blocks, effectively improving the ability of feature representation. Second, distinct attention modules are designed to integrate into different layers of the network to fully exploit contextual information of the source images, and we leverage the total loss to guide the fusion process to focus on vital regions and compensate for missing information. Extensive qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art methods in both visual effects and evaluation metrics. The experimental results on public datasets show that our method can improve the entropy (EN) by 4.80%, standard deviation (SD) by 3.97%, correlation coefficient (CC) by 1.86%, correlations of differences (SCD) by 9.98%, and multi-scale structural similarity (MS_SSIM) by 5.64%, respectively. In addition, experiments with the VIFB dataset further indicate that our approach outperforms other comparable models.Keywords
Funding Information
- National Key Research and Development Program of China (2020YFC0833406)
- National Natural Science Foundation of China (62102112)
- Basic Research Plan of Guizhou Province (Qiankehejichu-ZK[2021]Yiban310)
- Guizhou Provincial Science and Technology Projects (QKHJCZK2022YB19)
- Guizhou Provincial Science and Technology Projects (QKHJCZK2023YB143)
- Youth Science and Technology Talents Cultivating Object of Guizhou Province (QJHKY2021104)
This publication has 51 references indexed in Scilit:
- Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filtersInformation Fusion, 2016
- An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensingInfrared Physics & Technology, 2016
- A new image quality metric for image fusion: The sum of the correlations of differencesAEU - International Journal of Electronics and Communications, 2015
- A general framework for image fusion based on multi-scale transform and sparse representationInformation Fusion, 2015
- A Review of Quality Metrics for Fused ImageAquatic Procedia, 2015
- Image fusion based on pixel significance using cross bilateral filterSignal, Image and Video Processing, 2013
- Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transformSignal, Image and Video Processing, 2012
- Assessment of image fusion procedures using entropy, image quality, and multispectral classificationJournal of Applied Remote Sensing, 2008
- In-fibre Bragg grating sensorsMeasurement Science and Technology, 1997
- Image quality measures and their performanceIEEE Transactions on Communications, 1995