Learning Spectral Cues for Multispectral and Panchromatic Image Fusion
- 2 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 6964-6975
Abstract
Recently, deep learning based multispectral (MS) and panchromatic (PAN) image fusion methods have been proposed, which extracted features automatically and hierarchically by a series of non-linear transformations to model the complicated imaging discrepancy. But they always pay more attention to the extraction and compensation of spatial details and use the mean squared error or mean absolute error as a loss function, regardless of the preservation of spectral information contained in multispectral images. For the sake of the improvements in both spatial and spectral resolution, this paper presents a novel fusion model that takes the spectral preservation into consideration, and learns the spectral cues from the process of generating a spectrally refined multispectral image, which is constrained by a spectral loss between the generated image and the reference image. Then these spectral cues are used to modulate the PAN features to obtain final fusion result. Experimental results on reduced-resolution and full-resolution datasets demonstrate that the proposed method can obtain a better fusion result in terms of visual inspection and evaluation indices when compared with current state-of-the-art methods.Keywords
Funding Information
- National Natural Science Foundation of China (62201467, U19B2037)
- Basic and Applied Basic Research Foundation of Guangdong Province (2021A1515110544)
- Natural Science Basic Research Program of Shaanxi (2022JQ-686)
This publication has 52 references indexed in Scilit:
- SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified FrameworkIEEE Transactions on Image Processing, 2015
- A Critical Comparison Among Pansharpening AlgorithmsIEEE Transactions on Geoscience and Remote Sensing, 2014
- A New Pansharpening Method Based on Spatial and Spectral Sparsity PriorsIEEE Transactions on Image Processing, 2014
- A Regularized Model-Based Optimization Framework for Pan-SharpeningIEEE Transactions on Image Processing, 2014
- Information fusion techniques for change detection from multi-temporal remote sensing imagesInformation Fusion, 2013
- Image fusion based on a new contourlet packetInformation Fusion, 2010
- Multispectral and Panchromatic Data Fusion Assessment Without ReferencePhotogrammetric Engineering & Remote Sensing, 2008
- Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan DataIEEE Transactions on Geoscience and Remote Sensing, 2007
- A Global Quality Measurement of Pan-Sharpened Multispectral ImageryIEEE Geoscience and Remote Sensing Letters, 2004
- Image fusion—the ARSIS concept and some successful implementation schemesISPRS Journal of Photogrammetry and Remote Sensing, 2003