MHIAIFormer: Multihead Interacted and Adaptive Integrated Transformer With Spatial-Spectral Attention for Hyperspectral Image Classification

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Deep learning is an effective method for hyperspectral image (HSI) classification, where CNN-based and Transformer-based methods have achieved excellent performance. However, there are some drawbacks to the existing CNN-based and Transformer-based HSI classification approaches: 1) CNN-based methods are deficient in showing the extraction of multiscale features and localized features owing to the fixed-size input patch. 2) the MHSA module ignores the interaction capability between multiple attention heads, which leads to insufficient feature fusion in various directions. 3) The weights of attention heads in various directions are disregarded in the MHSA and attention heads are simply concatenated horizontally. To address the above limitations, a novel Multi-Head Interacted and Adaptive Integrated Transformer (MHIAIFormer) with Spatial-Spectral Attention which integrates the respective advantages of convolutions and transformers is proposed in this study. A pyramidal spatial-spectral attention feature extraction (PS2A) module is adopted to efficiently capture the localized and multiscale feature information of HSI. The output of PS2A is then sent to the transformer encoder stage through a grouped multiscale cross-dimension (GMCD) embedding module, which includes additive self-attention using multi-head interaction and MHSA with adaptive multi-head merging to capture the long-range dependencies of the features. Extensive experiments on four datasets verify that our proposed approach achieves more satisfactory classification accuracy when compared with state-of-the-art models. The overall accuracy (OA) of the proposed model achieved 95.97%, 98.68%, 92.68%, and 99.49% on four datasets.
Funding Information
  • Central Guiding Local Science and Technology Development Fund of Shandong-Yellow River Basin Collaborative Science and Technology Innovation Special Project (YDZX2023019)
  • Natural Science Foundation of Shandong Province (2018GNC110025, ZR2020QF067)

This publication has 57 references indexed in Scilit: