ELS2T: Efficient Lightweight Spectral–Spatial Transformer for Hyperspectral Image Classification

Authors:
Publisher website
Google Scholar
Abstract
In recent years, convolutional neural networks (CNNs) have been extensively used in hyperspectral image (HSI) classification tasks and achieved desirable performance. However, CNNs expand the receptive field by stacking convolutional layers and pooling layers, but the actual receptive field is still insufficient, so it is hard to capture the global representations of HSIs. In addition, the existing CNN models used for HSI classification have low computational efficiency, and can not effectively fuse spectral and spatial information. To alleviate the above limitations, this paper proposes a novel efficient lightweight spectral-spatial transformer (ELS2T) specially designed for HSI classification. The transformer model can model the long-distance feature dependencies in the HSI cube. First, we design a global multiscale attention module (GMAM) to effectively highlight the useful information and weaken the useless information. Second, considering the different importance of spectral and spatial information for classification tasks, an adaptive feature fusion module (AFFM) is proposed to adaptively fuse the acquired spectral and spatial information. Finally, to improve the computational efficiency, we design the lightweight separable spatial-spectral self-attention (S 3 A) module to replace the multi-head self-attention (MHSA) module in the transformer encoder. Experimental results on the four well-known hyperspectral datasets show that our model is superior to the other state-of-the-art deep learning methods in both computational efficiency and classification performance.
Funding Information
  • Natural Science Foundation of Shandong Province (No. ZR2020QF067, No.2018GNC110025)
  • ?Taishan Scholar? Project of Shandong Province (No. TSXZ201712)
  • National Natural Science Foundation of China (No. 41871253)

This publication has 55 references indexed in Scilit: