Lei Tong

The joint classification of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data holds immense potential by leveraging their complementary nature. However, effectively extracting and adaptively fusing their heterogeneous, multiscale features while simultaneously modeling complex long-range dependencies remains a significant challenge for existing methods. To address this challenge, we propose a novel framework, the Hierarchical MultiScale Features Mamba (HMSFMamba), for joint HSI-LiDAR classification. HMSFMamba first employs a hybrid convolutional residual module to extract multiscale spatial features. It then leverages a Mamba-based module to both enhance HSI-LiDAR spatial interaction through an optimized scanning mechanism and filter HSI spectral features using a cascaded attention mechanism. Finally, an attention-gated fusion module adaptively fuses these multimodal features to achieve robust classification. Extensive experiments on three benchmark datasets demonstrate that HMSFMamba consistently outperforms state-of-the-art methods.

One of the key challenges in cross-domain few-shot hyperspectral image classification lies in effectively leveraging spectral-spatial features while mitigating semantic inconsistencies between domains. To address this, we propose a Multi-Level Style and Semantic Enhancement (MSS-CFSL) framework, which enhances both low-level style features and high-level semantic representations to improve domain adaptation. Specifically, MSS-CFSL dynamically aligns spatial and spectral features using 2D spatial and 3D spectral style information from the source domain, ensuring better feature consistency in the target domain. Furthermore, a learnable class-level shared representation is introduced to bridge the semantic gap, enhancing the model’s ability to transfer discriminative knowledge under limited semantic annotations. Additionally, MSS-CFSL integrates a spectral-spatial knowledge fusion mechanism with adaptive feature enhancement, balancing the extraction of structural and spectral band information. Extensive experiments on four hyperspectral datasets show that MSS-CFSL improves the overall accuracy on the four datasets by 11.15%, 12.82%, 5.04%, and 4.42%, respectively, when compared to eight state-of-the-art cross-domain few-shot methods, underscoring its effectiveness in cross-domain few-shot scenarios.

Labeling hyperspectral image (HSI) data is time-consuming and laborious, resulting in limited labeled samples for training deep learning-based classifiers. To address this challenge, we propose ADLKAN, a cross-domain few-shot learning framework based on an adversarial dilated large kernel attention network. First, the ADLKAN consists of an adversarial dilated large kernel attention feature extractor, which can enhance the ability to capture sparse features. The feature extractor includes two cooperative spectral-spatial attention blocks with large kernel convolution, each of which extracts spectral and spatial features separately. These features are then fused to capture local contextual information and long-range dependencies in the spatial and spectral domains with attention networks. Second, ADLKAN also designs a multi-scale conditional domain discriminator to alleviate the weakness of few-shot learning under domain shift. The detailed multi-scale features are extracted by three sub-discriminators on different input scales. Finally, Adversarial training is then adopted to optimize the embedding feature extractor and strengthen its consistency in uncertain regions of samples from different domains. Experimental results on five public HSI datasets show that the proposed ADLKAN outperforms other state-of-the-art methods.

Hyperspectral image (HSI) classification is an essential task in remote sensing, but its performance is greatly affected by limited labeled samples. Currently, generative adversarial networks (GANs) based methods can generate the virtue samples to augment the training set. However, with limited labeled data, GANs perform poorly in capturing features during sample generation. Very few relation networks (RN) and few-shot learning methods considered data augmentation to enhance performance. To address this challenge, we propose FSHyperRGAN, a few-shot HSI classification method based on relational generative adversarial network, which uses GANs to augment the training samples for relation networks, while leveraging relation feature extraction to guide the generation of specific class samples. FSHyperRGAN comprises four modules: a data processing module converting the HSI data to 1D and 3D features, an adversarial generation module synthesizing virtual samples conditioned on labels, a data embedding and reconstruction module encoding latent spaces for accurate sample reconstruction while preserving category characteristics, and a relation computation module computing relation scores across generated, reconstructed, and original samples. In addition, a relational feature matching scheme is also applied, which can use virtual samples to guide classification. Two FSHyperRGAN frameworks are designed, 1D-FSHyperRGAN and 3D-FSHyperRGAN, which can be utilized for 1D spectral or 3D spatial-spectral classification. Experiments on widely used HSI data sets illustrate that the proposed method outperforms several state-of-the-art methods and achieves overall accuracies of 92.64%, 86.82%, 83.64%, and 84.57% on KSC, PaviaU, Houston, and WHU-Hi-HongHu data sets, respectively.

While deep learning has been widely used in the hyperspectral image (HSI) classification, lacking labeled HSI poses significant challenges to effective and sufficient learning. To address this issue, this letter introduces a Prototypical Network with Residual Capsule for few-shot HSI classification (PN-ResCapsNet). Compared with the CNN, the capsule networks can better capture spatial relationships. To better extract HSI features, residual structures, and self-attention mechanisms are incorporated, which can overcome the limitation of shallow feature extraction in capsule networks. Moreover, a bias-reduction (BR) method, consisting of an inter-class bias-reduction module and an intra-class bias-reduction module, is designed to rectify the prototypes, which can mitigate the bias between the support and the query set and alleviate the bias between the calculated prototype and the expected prototype. Experiments on widely used HSI datasets illustrate that the proposed method outperforms several state-of-the-art methods and achieves overall accuracies of 92.07%, 81.02%, and 85.35% on KSC, HT, and PU datasets, respectively.

Due to its ability to capture long-range dependencies, self-attention mechanism based transformer models are introduced for hyperspectral image classification. However, the self-attention mechanism has only spatial adaptability but ignores channel adaptability, thus cannot well extract complex spectral-spatial information in hyperspectral images. To tackle this problem, in this paper, we propose a novel spectral-spatial large kernel attention network (SSLKA) for hyperspectral image classification. SSLKA consists of two consecutive cooperative spectral-spatial attention blocks with large convolution kernels, which can efficiently extract features in spectral and spatial domains simultaneously. In each cooperative spectral-spatial attention block, we employ the spectral attention branch and the spatial attention branch to generate the attention maps, respectively, and then fuse the extracted spatial features with the spectral features. With large kernel attention, we can enhance the classification performance by fully exploiting local contextual information, capturing long-range dependencies, as well as be adaptive in the channel dimension. Experimental results on widely used benchmark datasets show that our method achieves higher classification accuracy in terms of overall accuracy, average accuracy, and Kappa than several state-of-the-art methods.

In recent years, deep learning has been widely applied to hyperspectral image (HSI) classification with great success. However, since labeling hyperspectral images is time-consuming and labor-intensive, a limited number of labeled hyperspectral images are available, making it difficult to train feature extractors and classifiers. To address this challenge, this paper introduces a self-supervised spectral-spatial graph prototypical network for few-shot HSI classification (S ⁴ GPN). In addition, we combine self-supervised learning with few-shot learning to provide additional semantic information to improve classification accuracy. Our method consists of three stages, including the prototype network (PN) stage, the self-supervised learning (SSL) stage, and the fusion stage (Fusion stage), with each stage progressively improving the classification performance. In the PN stage, we perform supervised learning using a prototype network structure and leverage the supervised information to guide self-supervised learning. In the SSL stage, we use the SimSiam structure in a novel way for model training after the augmentation of spectral and spatial data separately. Finally, the features learned in the first two stages are fused to improve the quality of the feature representation. These three stages use the same structured feature extractors. To extract more diverse and discriminative feature representations for the HSI classification task, our method uses the graph convolution network (GCN) and the dense network (Densenet) to extract spectral information and spatial information, respectively. Experiments on four data sets and comparisons with the state-of-the-art methods demonstrate that our proposed S ⁴ GPN outperforms other methods for HSI classification.

Recently, convolutional neural networks (CNNs) have demonstrated impressive capabilities in the representation and classification of hyperspectral remote sensing images. Traditional CNNs require massive data to sufficiently train the network. To tackle this problem, graph convolutional network (GCN) has been introduced for hyperspectral image classification. GCN methods usually construct the graph from either spectral or spatial domain, which has not adequately explored the information in the joint spectral–spatial domain. In this article, we propose a superpixel spectral–spatial feature fusion graph convolution network for hyperspectral image classification (S3FGCN). S3FGCN can comprehensively use information in spectral, spatial, and spectral–spatial domains with limited data. Moreover, to enhance the performance, we explore a shared weights’ GCN in the spectral–spatial domain. To further improve the efficiency, superpixels are used to construct the adjacency matrix. Finally, dynamic sampling is adopted to make the model focus more on difficult samples. In the experiments on four datasets, S3FGCN demonstrates better accuracy compared with the state-of-the-art hyperspectral image classification methods.

Hyperspectral classification is an important technique for remote sensing image analysis. For the current classification methods, limited training data affect the classification results. Recently, Conditional Variational Autoencoder Generative Adversarial Network (CVAEGAN) has been used to generate virtual samples to augment the training data, which could improve the classification performance. To further improve the classification performance, based on the CVAEGAN, we propose a Self-Attention-Based Conditional Variational Autoencoder Generative Adversarial Network (SACVAEGAN). Compared with CVAEGAN, we first use random latent vectors to obtain more enhanced virtual samples, which can improve the generalization performance. Then, we introduce the self-attention mechanism into our model to force the training process to pay more attention to global information, which can achieve better classification accuracy. Moreover, we explore model stability by incorporating the WGAN-GP loss function into our model to reduce the mode collapse probability. Experiments on three data sets and a comparison of the state-of-art methods show that SACVAEGAN has great advantages in accuracy compared with state-of-the-art HSI classification methods.

The convolutional neural network (CNN) is a feed-forward neural network with deep structure and convolution operation. In the hyperspectral image (HSI) classification, CNN has demonstrated excellent performance in extracting spectral and spatial information. However, the inherent complexity and high dimension of HSIs still limit the performance of most neural network models. The powerful feature extraction ability of CNN is normally achieved by dozens or more layers, which brings a series of problems such as gradient vanishing, overfitting, and slow training speed. In order to address these problems, this article presents a CNN architecture-based stochastic depth residual network (SDRN), which is specially designed for HSI data. This model takes the original 3-D cube as the input and 3-D convolution is used to extract abundant spectral and spatial features through corresponding residual blocks. In order to reduce the training time, we adopt a stochastic depth strategy. For each small batch, a sublayer is randomly discarded by an identity function. During the testing stage, the residual network with complete depth is used. Experiments on three datasets and a comparison of the state-of-art methods show that SDRN has great advantages in accuracy and training time compared with state-of-the-art HSI classification methods.