Multiscale Spatial-Spectral CNN-Transformer Network for Hyperspectral Image Super-Resolution
Open Access
- 30 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 12116-12132
Abstract
Remarkable strides have been made in super-resolution methods based on deep learning for hyperspectral images (HSIs), which are capable of enhancing the spatial resolution. However, these methods predominantly focus on capturing local features using convolutional neural networks (CNNs), neglecting the comprehensive utilization of global spatial-spectral information. To address this limitation, we innovatively propose a Multi-scale Spatial-spectral CNN-transformer network for Hyperspectral image Super Resolution, namely MSHSR. MSHSR not only applies the local spatial-spectral characteristics but also innovatively facilitates the collaborative exploration and application of spatial details and spectral data globally. Specifically, we first design a multi-scale spatial-spectral fusion module (MSSFM), which integrates dilated-convolution parallel branches (DPB) and a hybrid spectral attention mechanism (HSAM) to address the strong local correlations in HSIs, effectively capturing and fusing multi-scale local spatial-spectral information. Furthermore, in order to fully exploit the global contextual consistency in HSIs, we introduce a sparse spectral transformer module (SSTM). This module processes the previously obtained local spatial-spectral features, thoroughly exploring the elaborate global inter-relationship and long-range dependencies among different spectral bands through a coarse-to-fine strategy. Extensive experimental results on three hyperspectral datasets demonstrate the superior performance of our method, outperforming comparison methods both in quantitative metrics and visual performance.Keywords
Funding Information
- National Natural Science Foundation of China (42271140, 62403379)
- National Key Research and Development Program of China (2023YFF0715103)
- Key Research and Development Program of Shaanxi Province of China (2023-YBGY-242, 2024GX-YBXM-149)
- Xi'an Science and Technology (23ZDCYJSGG0009-2023)
This publication has 57 references indexed in Scilit:
- Deep Learning-Based Single-Image Super-Resolution: A Comprehensive ReviewIEEE Access, 2023
- An efficient unfolding network with disentangled spatial-spectral representation for hyperspectral image super-resolutionInformation Fusion, 2023
- ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and BeyondInternational Journal of Computer Vision, 2023
- Pre-Trained Image Processing Transformer2021
- Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net2019
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks2018
- Single Hyperspectral Image Super-Resolution with Grouped Deep Recursive Residual Network2018
- Acknowledgement of reviewer services to the International Journal Applied Earth Observation and GeoinformationInternational Journal of Applied Earth Observation and Geoinformation, 2017
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- A comparative study of spatial approaches for urban mapping using hyperspectral ROSIS images over Pavia City, northern ItalyInternational Journal of Remote Sensing, 2009