Dual-Collaborative Fusion Model for Multispectral and Panchromatic Image Fusion
- 29 December 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-15
Abstract
The aim of multispectral (MS) and panchromatic (PAN) image fusion is to obtain an MS image that has high resolution in both spectral and spatial domains. During the fusion process, there are two important issues, i.e., spectral information preservation and spatial information enhancement. In this article, we propose a dual-collaborative fusion model that considers not only the spectral correlation collaboration but also the spatial-spectral collaboration. First, the features of PAN and MS images are extracted by a shared feature embedding network. Then, in order to enhance the spatial details, the PAN features are decomposed into four subbands, and the collaborative relationships among subbands are fully explored to refine the features. After the refinement of the subbands, the high-frequency components are directly taken as the inputs of the reconstruction network, while the low-frequency components are transformed by the guidance generation network to accomplish the spatial-spectral collaboration and also make preparations for the spectral adjustment. To explore the spectral correlation collaboration, a novel graph convolutional network is designed for the modulation of intraspectral relationships. Finally, the adjusted MS features are combined with the high-frequency components of PAN features to reconstruct the high-resolution MS image. Experimental results show that the proposed method outperforms traditional state-of-the-art pan-sharpening methods as well as the available deep learning-based ones.Keywords
Funding Information
- National Natural Science Foundation of China (61771380, U1701267, 61906145, U1730109, 91438103)
- Major Research Plan in Shaanxi Province of China (2017ZDXM-GY-103, 2017ZDCXL-GY-03-02l)
- Science Basis Research Program in Shaanxi Province of China (16JK1823, 2017JM6086, 2019JQ-663)
- China Postdoctoral Science Foundation (2018M643587)
This publication has 46 references indexed in Scilit:
- A New Pansharpening Method Based on Spatial and Spectral Sparsity PriorsIEEE Transactions on Image Processing, 2014
- An Improved Adaptive Intensity–Hue–Saturation Method for the Fusion of Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters, 2013
- Two-Step Sparse Coding for the Pan-Sharpening of Remote Sensing ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013
- Image fusion based on a new contourlet packetInformation Fusion, 2010
- Multispectral and Panchromatic Data Fusion Assessment Without ReferencePhotogrammetric Engineering & Remote Sensing, 2008
- Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan DataIEEE Transactions on Geoscience and Remote Sensing, 2007
- Remote sensing image fusion using the curvelet transformInformation Fusion, 2007
- A Global Quality Measurement of Pan-Sharpened Multispectral ImageryIEEE Geoscience and Remote Sensing Letters, 2004
- A Fast Intensity–Hue–Saturation Fusion Technique With Spectral Adjustment for IKONOS ImageryIEEE Geoscience and Remote Sensing Letters, 2004
- Comparison of Three Different Methods to Merge Multiresolution and Multispectral Data: LANDSAT TM and SPOT Panchromatic: ABSTRACTAAPG Bulletin, 1990