Abstract
Remarkable strides have been made in super-resolution methods based on deep learning for hyperspectral images (HSIs), which are capable of enhancing the spatial resolution. However, these methods predominantly focus on capturing local features using convolutional neural networks (CNNs), neglecting the comprehensive utilization of global spatial-spectral information. To address this limitation, we innovatively propose a Multi-scale Spatial-spectral CNN-transformer network for Hyperspectral image Super Resolution, namely MSHSR. MSHSR not only applies the local spatial-spectral characteristics but also innovatively facilitates the collaborative exploration and application of spatial details and spectral data globally. Specifically, we first design a multi-scale spatial-spectral fusion module (MSSFM), which integrates dilated-convolution parallel branches (DPB) and a hybrid spectral attention mechanism (HSAM) to address the strong local correlations in HSIs, effectively capturing and fusing multi-scale local spatial-spectral information. Furthermore, in order to fully exploit the global contextual consistency in HSIs, we introduce a sparse spectral transformer module (SSTM). This module processes the previously obtained local spatial-spectral features, thoroughly exploring the elaborate global inter-relationship and long-range dependencies among different spectral bands through a coarse-to-fine strategy. Extensive experimental results on three hyperspectral datasets demonstrate the superior performance of our method, outperforming comparison methods both in quantitative metrics and visual performance.
Funding Information
  • National Natural Science Foundation of China (42271140, 62403379)
  • National Key Research and Development Program of China (2023YFF0715103)
  • Key Research and Development Program of Shaanxi Province of China (2023-YBGY-242, 2024GX-YBXM-149)
  • Xi'an Science and Technology (23ZDCYJSGG0009-2023)

This publication has 57 references indexed in Scilit: