SSMSFuse: A Spectral and Spatial Multiscale Coupling Fusion Model for Hyperspectral and Multispectral Image
Open Access
- 4 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 17310-17327
Abstract
Hyperspectral image (HSI) has more spectral information than conventional images, which helps to distinguish targets in a complex scene more accurately. However, HSI typically has a low spatial resolution, which limits their application scenarios. To achieve high-resolution HSI (HR-HSI), we propose a spectral and spatial multi-scale coupling fusion model—SSMSFuse for hyperspectral and multispectral image (MSI). SSMSFuse couples the spatial information of MSI and the spectral information of HSI at multi-scales by means of a twobranch network structure, thus obtaining the fused images with high spatial and spectral resolution. SSMSFuse consists of two branches, namely the spatial embedding network (Spa-Net) and the spectral embedding network (Spe-Net). Spa-Net is constructed using a multi-scale convolutional neural network (CNN) to better mine multi-level spatial features from MSI. SpeNet is constructed using self-attention, which can model the longdistance spectral dependencies of HSI to better extract spectral information from HSI. Finally, to achieve interactive coupling of dual branch information, we designed a spatial-spectral guidance fusion block (SSGFB) to fuse features at different scales to avoid loss of spatial and spectral details. Experiments are carried out on four public datasets, and the results show that the proposed method can effectively improve the objective indicators of the fusion results, such as the PSNR, which is increased by 1.36%, and the RMSE, which is increased by 9.72% on the CAVE dataset, and satisfactory subjective results are also obtained. The source code is available at: https://github.com/cvmdsp/SSMSFuse.Keywords
Funding Information
- National Natural Science Foundation of China (62172139)
- Hebei Province Science and Technology Support Program (246Z0104G)
- Higher Education Science and Technology Research Project of Hebei Province (CXY2024031)
- Hebei University Research and Innovation Team Support Project (IT2023B05)
This publication has 53 references indexed in Scilit:
- Spatial–spectral unfolding network with mutual guidance for multispectral and hyperspectral image fusionPattern Recognition, 2024
- Reciprocal transformer for hyperspectral and multispectral image fusionInformation Fusion, 2023
- Learning the external and internal priors for multispectral and hyperspectral image fusionScience China Information Sciences, 2023
- MCT-Net: Multi-hierarchical cross transformer for hyperspectral and multispectral image fusionKnowledge-Based Systems, 2023
- HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening2022
- MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction2022
- NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results2022
- Hyperspectral Super-Resolution by Coupled Spectral Unmixing2015
- Bi-cubic interpolation for shift-free pan-sharpeningISPRS Journal of Photogrammetry and Remote Sensing, 2013
- Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial detailsInternational Journal of Remote Sensing, 2000