Enhanced Spatial-Frequency Synergistic Network for Multispectral and Hyperspectral Image Fusion
- 14 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-16
Abstract
Multispectral and hyperspectral image fusion (MHIF) seeks to combine high-resolution multispectral images (HR-MSIs) with low-resolution hyperspectral images (LR-HSIs) to create high-resolution hyperspectral images (HR-HSIs). Transformer-based architectures have recently become prominent in MHIF tasks due to their effective global self-attention mechanisms. However, the quadratic computational complexity of the global self-attention in Transformers presents significant challenges for practical applications. In this paper, we propose an enhanced spatial-frequency synergistic (ESFS) approach that leverages both spatial and frequency domain features to enhance fusion quality. Our ESFS framework introduces the condensed spatial augmentation module (CSAM), which condenses window features and employs cross-attention to balance extensive contextual understanding and detailed local feature extraction while reducing computational overhead. Additionally, we develop the selective frequency decomposition module (SFDM), which utilizes global filters composed of phase and amplitude information in the frequency domain to retain features, effectively capturing deep frequency domain characteristics and their interdependencies. Comprehensive experiments on three benchmark MHIF datasets demonstrate that our method achieves superior performance, establishing a new state-of-the-art (SOTA) in both quantitative metrics and visual quality assessments. The code is available at http://szu-hsilab.com/.Keywords
Funding Information
- National Natural Science Foundation of China (42271336, 62271327, 42301375)
- Natural Science Foundation of Guangdong Province (2024A1515011079, 2022A1515011290, 2022A1515110076)
- Project of Department of Education of Guangdong Province (2023KCXTD029)
- Shenzhen Science and Technology Program (RCJC20221008092731042, JCYJ20220818100206015, KQTD20200909113951005, JCYJ20240813141635047)
This publication has 56 references indexed in Scilit:
- Unsupervised multi-level spatio-spectral fusion transformer for hyperspectral image super-resolutionOptics & Laser Technology, 2024
- Frequency Integration and Spatial Compensation Network for infrared and visible image fusionInformation Fusion, 2024
- Reciprocal transformer for hyperspectral and multispectral image fusionInformation Fusion, 2023
- Learning a 3D-CNN and Transformer prior for hyperspectral image super-resolutionInformation Fusion, 2023
- Multiscale spatial–spectral transformer network for hyperspectral and multispectral image fusionInformation Fusion, 2023
- Swin-Spectral Transformer for Cholangiocarcinoma Hyperspectral Image Segmentation2021
- Deep Residual Convolutional Neural Network for Hyperspectral Image Super-Resolution2017
- Densely Connected Convolutional Networks2017
- Bayesian sparse representation for hyperspectral image super resolution2015
- Statistics of real-world hyperspectral images2011