Hierarchical Unified Spectral-Spatial Aggregated Transformer for Hyperspectral Image Classification
Authors:
- 21 August 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3041-3047
Abstract
Vision Transformer (ViT) has recently been introduced into the computer vision (CV) field with its self-attention mechanism and gotten remarkable performance. However, simply applying ViT for hyperspectral image (HSI) classification is not applicable due to 1) ViT is a spatial-only self-attention model, but rich spectral information exists in HSI; 2) ViT needs sufficient training samples, but HSI suffers from limited samples; 3) ViT does not well learn local features; 4) multi-scale features for ViT are not considered. Furthermore, the methods which combine convolutional neural network (CNN) and ViT generally suffer from a large computational burden. Hence, this paper tends to design a suitable pure ViT based model for HSI classification as the following points: 1) spectral-only vision transformer with all tokens’ aggregation; 2) spatial-only local-global transformer; 3) cross-scale local-global feature fusion, and 4) a cooperative loss function to unify the spectral and spatial features. As a result, the proposed idea achieves competitive classification performance on three public datasets than other state-of-the-art methods.Keywords
This publication has 30 references indexed in Scilit:
- Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the ArtIEEE Geoscience and Remote Sensing Magazine, 2017
- Recent Advances on Spectral–Spatial Hyperspectral Image Classification: An Overview and New GuidelinesIEEE Transactions on Geoscience and Remote Sensing, 2017
- Multi-scale 3D deep convolutional neural network for hyperspectral image classification2017
- Densely Connected Convolutional Networks2017
- Feature Pyramid Networks for Object Detection2017
- Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural NetworksIEEE Transactions on Geoscience and Remote Sensing, 2016
- Deep Residual Learning for Image Recognition2016
- Deep Learning-Based Classification of Hyperspectral DataIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014
- Mineral mapping of makhtesh ramon in israel using hyperspectral remote sensing day and night LWIR images2014
- Investigation of epifauna coverage on seagrass blades using spatial and spectral analysis of hyperspectral images2013