UGSA-Net: Uncertainty-Gated Attention and Self-Supervised Alignment for Calibration-Aware Hyperspectral Image Classification
Open Access
Abstract
Uniform spatial weighting in patch-based hyperspectral classifiers indiscriminately aggregates spectral information, even when individual pixels are already class-diagnostic, degrading accuracy and interpretability. We propose UGSA-Net, an ambiguity-guided spatial–spectral transformer that infers a per-pixel ambiguity field without direct label supervision. A lightweight ambiguity head maps each token to a scalar in , trained end-to-end with no target of its own, so that the field is shaped only by the training losses that reach it through the token mixer. This field dually modulates the token mixer: it biases query and key projections with a spectral angle mapper (SAM) prior to emphasise spectrally similar patches, and rescales token outputs to amplify contributions from ambiguous locations. The value pathway remains unmodulated, so the network reweights spatial information without distorting the spectral content entering the mixture. The ambiguity head and the classification backbone are trained end-to-end, following a self-supervised latent-alignment stage that uses no labels; model selection jointly scores overall accuracy (OA), few-shot accuracy and expected calibration error (ECE) on validation data. Across six benchmark scenes from four sensors, under a stratified pixel-level split in which training and test windows overlap, UGSA-Net exceeds 98.9% OA on every scene over three runs, and on Indian Pines it is comparable to competing architectures retrained under the same protocol. These values are upper bounds rather than estimates for new scenes: on two of the six scenes, almost no labelled pixel has a competing class inside the input window. A single validation-tuned temperature reduces ECE from 0.24–0.28 to below 0.016 without changing any prediction, revealing that residual miscalibration is governed by the decision head rather than by the data representations. The benefit of ambiguity gating emerges when labels are scarce: with a reduced training set on Indian Pines, gating raised OA over the ungated model on all three partitions, in either direction, whereas, with the full training set, component effects fall within run-to-run variation.
Keywords
This publication has 24 references indexed in Scilit:
- Hyperspectral image classification via Manhattan self-attention transformer and adaptive global-local channel attentionJournal of King Saud University Computer and Information Sciences, 2026
- ATN-Hybrid: a hybrid attention network with deterministic-probabilistic mechanism for hyperspectral image classificationGeo-spatial Information Science, 2025
- Adaptive SimSiam Self-Supervised Networks with Hybrid PIO for Hyperspectral Image Classification2025
- Cross-scene hyperspectral image classification based on cross-domain feature extraction and category decision collaborative optimizationExpert Systems with Applications, 2025
- A survey of uncertainty in deep neural networksArtificial Intelligence Review, 2023
- Masked Vision Transformers for Hyperspectral Image Classification2023
- Weighted residual self-attention graph-based transformer for spectral–spatial hyperspectral image classificationInternational Journal of Remote Sensing, 2023
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Deep learning classifiers for hyperspectral imaging: A reviewISPRS Journal of Photogrammetry and Remote Sensing, 2019
- The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer dataRemote Sensing of Environment, 1993