Multiscale Spatial-Frequency Domain Dynamic Pansharpening of Remote Sensing Images Integrated With Wavelet Transform
- 19 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-15
Abstract
Pansharpening produces high-resolution multi-spectral (HRMS) images by enhancing the high-frequency details in multispectral (MS) images, with deep learning (DL) methods emerging as a new paradigm. However, most existing DL methods primarily focus on pansharpening in the spatial domain using convolutional layers with local kernels, which is not conducive to accurate reconstruction of global high frequencies, leading to spatial deformation and spectral distortion. To address this issue, we propose a bidirectional progressive spatial-frequency dynamic fusion network that constructs a multi-scale structure using wavelet transform. This network utilizes the wavelet transform’s ability to analyze frequency information in local neighborhoods to inject high frequencies from panchromatic (PAN) images into MS images. Additionally, it employs a wavelet-based adaptive fusion module to analyze spatial-spectral correlations and adaptively learn the injection gains for high-frequency details, thereby enhancing spectral fidelity. To further adapt to various scenarios, a dual-domain dynamic filtering block is proposed. It improves model generalization through data-specific dynamic filtering in both frequency and spatial domains, capturing global and local dependencies effectively. Quantitative metrics and visualization results on reduced and full-resolution images demonstrate that our method outperforms other advanced methods with low computational cost, producing high-fidelity HRMS images.Keywords
Funding Information
- National Key Research and Development Program of China (2020YFF0304104)
This publication has 59 references indexed in Scilit:
- Multi-scale Wavelet Transformer for Face Forgery Detection2023
- Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges aheadInformation Fusion, 2023
- Laplacian pyramid networks: A new approach for multispectral pansharpeningInformation Fusion, 2022
- A coarse-to-fine boundary refinement network for building footprint extraction from remote sensing imageryISPRS Journal of Photogrammetry and Remote Sensing, 2022
- A Global Context-aware and Batch-independent Network for road extraction from VHR satellite imageryISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challengesInformation Fusion, 2019
- Pansharpening of multispectral images using the nonseparable framelet lifting transform with high vanishing momentsInformation Fusion, 2014
- Remote sensing image fusion using the curvelet transformInformation Fusion, 2007
- A Variational Model for P+XS Image FusionInternational Journal of Computer Vision, 2006
- Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniquesRemote Sensing of Environment, 1987