Abstract
Hyperspectral image (HSI) has more spectral information than conventional images, which helps to distinguish targets in a complex scene more accurately. However, HSI typically has a low spatial resolution, which limits their application scenarios. To achieve high-resolution HSI (HR-HSI), we propose a spectral and spatial multi-scale coupling fusion model—SSMSFuse for hyperspectral and multispectral image (MSI). SSMSFuse couples the spatial information of MSI and the spectral information of HSI at multi-scales by means of a twobranch network structure, thus obtaining the fused images with high spatial and spectral resolution. SSMSFuse consists of two branches, namely the spatial embedding network (Spa-Net) and the spectral embedding network (Spe-Net). Spa-Net is constructed using a multi-scale convolutional neural network (CNN) to better mine multi-level spatial features from MSI. SpeNet is constructed using self-attention, which can model the longdistance spectral dependencies of HSI to better extract spectral information from HSI. Finally, to achieve interactive coupling of dual branch information, we designed a spatial-spectral guidance fusion block (SSGFB) to fuse features at different scales to avoid loss of spatial and spectral details. Experiments are carried out on four public datasets, and the results show that the proposed method can effectively improve the objective indicators of the fusion results, such as the PSNR, which is increased by 1.36%, and the RMSE, which is increased by 9.72% on the CAVE dataset, and satisfactory subjective results are also obtained. The source code is available at: https://github.com/cvmdsp/SSMSFuse.
Funding Information
  • National Natural Science Foundation of China (62172139)
  • Hebei Province Science and Technology Support Program (246Z0104G)
  • Higher Education Science and Technology Research Project of Hebei Province (CXY2024031)
  • Hebei University Research and Innovation Team Support Project (IT2023B05)