Hyperspectral Image Classification Based on a Locally Enhanced Transformer Network

Authors:
Publisher website
Google Scholar
Abstract
Recently, transformer-based models have achieved remarkable performance in the hyperspectral image (HSI) classification. However, due to the limited training data, existing methods often show limited capability of capturing fine-grained local features. Although attempts have been made to solve this problem, the large amount of parameters imposes the risk of overfitting. In this paper, we propose a locally enhanced transformer network for HSI classification with fewer network parameters, which mainly consists of a multi-branch spatial-spectral tokenization (MSST) module and a dual-branch transformer encoder (DTE) module. The MSST generates effective spatialspectral tokens through diverse convolutions with a residual connection. The DTE consists of a global transformer branch and a locally enhanced transformer branch, which are used to capture the global and local spatial dependencies of HSI, respectively. Unlike the conventional self-attention module used in the global branch, we propose an improved multi-head selfattention (IMSA) module in the local branch by incorporating the local prior information of HSI with graph convolution, to enhance the local information extraction. To fuse the global and local features from the two branches, we introduce an adaptive strategy by using learnable weights for both branches. We devise our MSST and DTE with a shallow architecture, significantly reducing the number of parameters. Experimental results on benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art.
Funding Information
  • National Natural Science Foundation of China (42301425)
  • China Postdoctoral Science Foundation (2023M743299)
  • “China University of Geosciences (CUG) Scholar” Scientific Research Funds at China University of Geosciences (2022164)
  • Fundamental Research Funds for the Central Universities, China University of Geosciences (CUG240628)

This publication has 67 references indexed in Scilit: