Xinyu Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Xinyu Zhang has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Licheng Jiao
Xidian University, Xian, China
7 shared publications - Shuyuan Yang
Xidian University
7 shared publications - Zhixi Feng
Xidian University
7 shared publications - Liangliang Song
School of Artificial Intelligence, Xidian University, Xian, China
6 shared publications - Yantao Wei
Central China Normal University
3 shared publications Huang Yao
Hubei Research Center for Educational Informationization, Faulty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China
1 shared publication
Latest Publications
- 10 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (9) , 8589-8601
Interactive Spectral-Spatial Transformer for Hyperspectral Image Classification
The abundant spectral signatures and spatial contexts are effectively utilized as the key to hyperspectral image (HSI) classification. Existing convolutional neural networks (CNNs), only focus on locally spatial context information and lack the ability to learn global spectral sequence representations, whereas the transformer performs well in learning the global dependence of sequential data. To solve this issue, inspired by the transformer, we propose an interactive global spectral and local spatial feature fusion transformer called ISSFormer. Specifically, we achieve an elegant integration of self-attention and convolution in a parallel design, i.e., the multi-head self-attention mechanism (MHSA) and the local spatial perception mechanism (LSP). ISSFormer can learn both local spatial feature representation and global spectral feature representation simultaneously. More significantly, we propose a bi-directional interaction mechanism (BIM) of features across the parallel branch to provide complementary clues. The local spatial features and the global spectral features interact through the BIM which could emphasize the local spatial details and add spatial constraints to overcome spectral variability, and can further improve classification performance. With extensive experiments on three benchmark datasets, including Indian Pines, Pavia University, and WHU-Hi-HanChuan, ISSFormer can accomplish superior classification accuracy and visualization performance.
- 10 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (9) , 8589-8601
Interactive Spectral-Spatial Transformer for Hyperspectral Image Classification
The abundant spectral signatures and spatial contexts are effectively utilized as the key to hyperspectral image (HSI) classification. Existing convolutional neural networks (CNNs), only focus on locally spatial context information and lack the ability to learn global spectral sequence representations, whereas the transformer performs well in learning the global dependence of sequential data. To solve this issue, inspired by the transformer, we propose an interactive global spectral and local spatial feature fusion transformer called ISSFormer. Specifically, we achieve an elegant integration of self-attention and convolution in a parallel design, i.e., the multi-head self-attention mechanism (MHSA) and the local spatial perception mechanism (LSP). ISSFormer can learn both local spatial feature representation and global spectral feature representation simultaneously. More significantly, we propose a bi-directional interaction mechanism (BIM) of features across the parallel branch to provide complementary clues. The local spatial features and the global spectral features interact through the BIM which could emphasize the local spatial details and add spatial constraints to overcome spectral variability, and can further improve classification performance. With extensive experiments on three benchmark datasets, including Indian Pines, Pavia University, and WHU-Hi-HanChuan, ISSFormer can accomplish superior classification accuracy and visualization performance.
- 6 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 4729-4744
Pseudo-Label-Assisted Subdomain Adaptation for Hyperspectral Image Classification
Cross-domain classification of hyperspectral data is a critical challenge in remote sensing, especially when labels are unavailable in the target domain. Deep learning-based domain adaptation (DA) methods have been widely used in recent years. However, curren methods primarily focus on the global domain structure of the source and target domains when considering domain adaptation, neglecting the subdomain structure within each class. Additionally, current methods directly employ predicted outputs without further exploring the confidence level of the target domain samples. These limitations lead to confusion in domain adaptation and hinder effective feature selection in neural networks. In this paper, we propose the Pseudo-Label-Assisted Subdomain Adaptation (PASDA) method, which addresses these limitations by jointly considering the subdomain structure of the source and target domains and adopting a sample selection strategy. PASDA aligns the subdomains while learning domain-invariant features as a foundation. Furthermore, it selects high-quality pseudo-labeled samples from the target domain to enhance the learning of domain-invariant features. For generating pseudo-labels in the target domain, we employ the Reweighted Pruning Label Propagation (RPLPA) strategy to reweight the output of the predicted target domain. Finally, the high-confidence samples with pseudo-labels are selected to finetune the network. The entropy regularized dual classifier constraint is introduced to enhance the discriminative feature extraction ability for the target domain. Extensive experiments on three public HSI cross-domain datasets, Pavia, Houston, and HyRANK, using overall accuracy (OA), average accuracy (AA) and kappa coefficient (Kappa) as the evaluation indicators of classification performance, demonstrate the superiority of our method. Compared with the existing state-of-the-art (SOTA) unsupervised domain adaptation (UDA) methods, our method improves OA by 2% and AA by 4%.
- 6 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 4729-4744
Pseudo-Label-Assisted Subdomain Adaptation for Hyperspectral Image Classification
Cross-domain classification of hyperspectral data is a critical challenge in remote sensing, especially when labels are unavailable in the target domain. Deep learning-based domain adaptation (DA) methods have been widely used in recent years. However, curren methods primarily focus on the global domain structure of the source and target domains when considering domain adaptation, neglecting the subdomain structure within each class. Additionally, current methods directly employ predicted outputs without further exploring the confidence level of the target domain samples. These limitations lead to confusion in domain adaptation and hinder effective feature selection in neural networks. In this paper, we propose the Pseudo-Label-Assisted Subdomain Adaptation (PASDA) method, which addresses these limitations by jointly considering the subdomain structure of the source and target domains and adopting a sample selection strategy. PASDA aligns the subdomains while learning domain-invariant features as a foundation. Furthermore, it selects high-quality pseudo-labeled samples from the target domain to enhance the learning of domain-invariant features. For generating pseudo-labels in the target domain, we employ the Reweighted Pruning Label Propagation (RPLPA) strategy to reweight the output of the predicted target domain. Finally, the high-confidence samples with pseudo-labels are selected to finetune the network. The entropy regularized dual classifier constraint is introduced to enhance the discriminative feature extraction ability for the target domain. Extensive experiments on three public HSI cross-domain datasets, Pavia, Houston, and HyRANK, using overall accuracy (OA), average accuracy (AA) and kappa coefficient (Kappa) as the evaluation indicators of classification performance, demonstrate the superiority of our method. Compared with the existing state-of-the-art (SOTA) unsupervised domain adaptation (UDA) methods, our method improves OA by 2% and AA by 4%.
- 9 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 20 (1545598X) , 1-5
Discrepant Bi-Directional Interaction Fusion Network for Hyperspectral and LiDAR Data Classification
In recent years, the joint classification approach of hyperspectral image (HSI) and light detection and ranging (LiDAR) data based on deep learning (DL) has received increasing attention. However, existing methods either lack interaction between heterogeneous features during feature extraction or treat them equally during the interaction, inevitably resulting in redundant information stacking and reaching the performance bottleneck. To this end, we propose a novel discrepant bi-directional interaction fusion network (DBIFNet) for the collaborative classification of HSI and LiDAR data. First, a discrepant bi-directional interaction module (DBDIM) is designed to establish correlations between heterogeneous features to enhance the respective feature learning. Furthermore, a cross-modal attention fusion module (CAFM) is developed to dynamically fuse multi-modal features, which can further improve classification performance. Extensive experiments on the Houston and Trento datasets demonstrate that the proposed DBIFNet can achieve competitive classification performance.
- 9 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 20 (1545598X) , 1-5
Discrepant Bi-Directional Interaction Fusion Network for Hyperspectral and LiDAR Data Classification
In recent years, the joint classification approach of hyperspectral image (HSI) and light detection and ranging (LiDAR) data based on deep learning (DL) has received increasing attention. However, existing methods either lack interaction between heterogeneous features during feature extraction or treat them equally during the interaction, inevitably resulting in redundant information stacking and reaching the performance bottleneck. To this end, we propose a novel discrepant bi-directional interaction fusion network (DBIFNet) for the collaborative classification of HSI and LiDAR data. First, a discrepant bi-directional interaction module (DBDIM) is designed to establish correlations between heterogeneous features to enhance the respective feature learning. Furthermore, a cross-modal attention fusion module (CAFM) is developed to dynamically fuse multi-modal features, which can further improve classification performance. Extensive experiments on the Houston and Trento datasets demonstrate that the proposed DBIFNet can achieve competitive classification performance.
- 19 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-13
Cross-Modal Contrastive Learning for Remote Sensing Image Classification
Recently, multi-modal remote sensing image (MRSI) classification has attracted increasing attention of researchers. However, classification of MRSI with limited labeled instances is still a challenging task. In this paper, a novel self-supervised cross-modal contrastive learning method is proposed for MRSI classification. Joint intra- and cross-modal contrastive learning are used to better mine multi-modal feature representations during pre-training, and the intra- and cross-modal contrastive learning objectives are jointly optimized, whereby it encourages the learned representation to be semantically consistent within and between modalities simultaneously. Moreover, a simple but effective hybrid cross-modal fusion module (HCFM) is designed in the fine-tuning stage, which could better compactly integrate complementary information across these modalities for more accurate classification. Extensive experiments are taken on four benchmark datasets (i.e., Houston 2013, Augsburg, Trento, and Berlin), and the results show that the proposed method outperforms state-of-the-art methods.
- 19 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-13
Cross-Modal Contrastive Learning for Remote Sensing Image Classification
Recently, multi-modal remote sensing image (MRSI) classification has attracted increasing attention of researchers. However, classification of MRSI with limited labeled instances is still a challenging task. In this paper, a novel self-supervised cross-modal contrastive learning method is proposed for MRSI classification. Joint intra- and cross-modal contrastive learning are used to better mine multi-modal feature representations during pre-training, and the intra- and cross-modal contrastive learning objectives are jointly optimized, whereby it encourages the learned representation to be semantically consistent within and between modalities simultaneously. Moreover, a simple but effective hybrid cross-modal fusion module (HCFM) is designed in the fine-tuning stage, which could better compactly integrate complementary information across these modalities for more accurate classification. Extensive experiments are taken on four benchmark datasets (i.e., Houston 2013, Augsburg, Trento, and Berlin), and the results show that the proposed method outperforms state-of-the-art methods.
- 4 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-16
Triple Contrastive Representation Learning for Hyperspectral Image Classification With Noisy Labels
Recently, Hyperspectral Image Classification (HIC) with noisy labels is attracting increasing interest. However, existing methods usually neglect to explore feature-dependent knowledge to reduce label noise, and thus perform poorly when the noise ratio is high or the clean samples are limited. In this paper, a novel Triple Contrastive Representation Learning (TCRL) framework is proposed from a deep clustering perspective for robust HIC with noisy labels. The TCRL explores the cluster-level, instance-level, and structure-level representation of HIC by defining triple learning loss. First, the strong and weak transformation are defined for hyperspectral data augmentation. Then, a simple yet effective lightweight Spectral Prior Attention-based Network (SPAN) is presented for spatial-spectral feature extraction of all augmented samples. Additionally, cluster-level and instance-level contrastive learning are performed on two projection subspaces for clustering and distinguishing samples respectively. Meanwhile, structure-level representation learning is employed to maximize the consistency of data after different projections. Taking the feature-dependent information learned by triple representation learning, our proposed end-to-end TCRL can effectively alleviate the overfitting of classifier to noisy labels. Extensive experiments have been taken on three public datasets with various noise ratios and two types of noise. The results show that the proposed TCRL could provide more robust classification results when training on noisy datasets compared with state-of-the-art methods, especially when clean samples are limited. The code will be available at https://github.com/Zhangxy1999.
- 4 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-16
Triple Contrastive Representation Learning for Hyperspectral Image Classification With Noisy Labels
Recently, Hyperspectral Image Classification (HIC) with noisy labels is attracting increasing interest. However, existing methods usually neglect to explore feature-dependent knowledge to reduce label noise, and thus perform poorly when the noise ratio is high or the clean samples are limited. In this paper, a novel Triple Contrastive Representation Learning (TCRL) framework is proposed from a deep clustering perspective for robust HIC with noisy labels. The TCRL explores the cluster-level, instance-level, and structure-level representation of HIC by defining triple learning loss. First, the strong and weak transformation are defined for hyperspectral data augmentation. Then, a simple yet effective lightweight Spectral Prior Attention-based Network (SPAN) is presented for spatial-spectral feature extraction of all augmented samples. Additionally, cluster-level and instance-level contrastive learning are performed on two projection subspaces for clustering and distinguishing samples respectively. Meanwhile, structure-level representation learning is employed to maximize the consistency of data after different projections. Taking the feature-dependent information learned by triple representation learning, our proposed end-to-end TCRL can effectively alleviate the overfitting of classifier to noisy labels. Extensive experiments have been taken on three public datasets with various noise ratios and two types of noise. The results show that the proposed TCRL could provide more robust classification results when training on noisy datasets compared with state-of-the-art methods, especially when clean samples are limited. The code will be available at https://github.com/Zhangxy1999.
- 9 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-13
Semantic Correntropy Representation for Hyperspectral Image Classification
Exploring semantic structure information contained in the hyperspectral image (HSI) is significant for accurate HSI classification (HSIC), especially when there are very few labeled samples. Besides, it is usually time-consuming to process HSI for many algorithms. In this paper, for a more accurate and efficient HSIC, a Semantic CorrEntropy Representation-based method (SCER) is presented. Specifically, superpixel segmentation is employed to capture structural information and improve computational efficiency. Each superpixel in HSI can be regarded as a semantic subregion. And for each semantic subregion, we define a semantic correntropy matrix to represent the intrinsic spectral similarity distribution, which can provide beneficial information for classification. However, the spatial sizes of diverse land covers in HSI have large variations, which means the rich semantic information cannot be fully captured by a single-scale segmentation map. Therefore, multiscale SCER (MSCER) with the multiscale segmentation strategy is proposed. Further, the features learned from multiple scales are integrated via multiple kernels to promote semantic information learning. The weights of multiple kernels can be determined adaptively. The semantic features of diverse land covers represented by MSCER have shown great intra-class compactness and inter-class separability. Compared with other state-of-the-art methods, our proposed MSCER has achieved competitive classification performance both in accuracy and efficiency with few labeled samples.
- 9 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-13
Semantic Correntropy Representation for Hyperspectral Image Classification
Exploring semantic structure information contained in the hyperspectral image (HSI) is significant for accurate HSI classification (HSIC), especially when there are very few labeled samples. Besides, it is usually time-consuming to process HSI for many algorithms. In this paper, for a more accurate and efficient HSIC, a Semantic CorrEntropy Representation-based method (SCER) is presented. Specifically, superpixel segmentation is employed to capture structural information and improve computational efficiency. Each superpixel in HSI can be regarded as a semantic subregion. And for each semantic subregion, we define a semantic correntropy matrix to represent the intrinsic spectral similarity distribution, which can provide beneficial information for classification. However, the spatial sizes of diverse land covers in HSI have large variations, which means the rich semantic information cannot be fully captured by a single-scale segmentation map. Therefore, multiscale SCER (MSCER) with the multiscale segmentation strategy is proposed. Further, the features learned from multiple scales are integrated via multiple kernels to promote semantic information learning. The weights of multiple kernels can be determined adaptively. The semantic features of diverse land covers represented by MSCER have shown great intra-class compactness and inter-class separability. Compared with other state-of-the-art methods, our proposed MSCER has achieved competitive classification performance both in accuracy and efficiency with few labeled samples.
- Open Access
- 23 June 2022
- Published by MDPI AG in Remote Sensing
- Vol. 14 (13) , 2997
Self-Supervised Assisted Semi-Supervised Residual Network for Hyperspectral Image Classification
Due to the scarcity and high cost of labeled hyperspectral image (HSI) samples, many deep learning methods driven by massive data cannot achieve the intended expectations. Semi-supervised and self-supervised algorithms have advantages in coping with this phenomenon. This paper primarily concentrates on applying self-supervised strategies to make strides in semi-supervised HSI classification. Notably, we design an effective and a unified self-supervised assisted semi-supervised residual network (SSRNet) framework for HSI classification. The SSRNet contains two branches, i.e., a semi-supervised and a self-supervised branch. The semi-supervised branch improves performance by introducing HSI data perturbation via a spectral feature shift. The self-supervised branch characterizes two auxiliary tasks, including masked bands reconstruction and spectral order forecast, to memorize the discriminative features of HSI. SSRNet can better explore unlabeled HSI samples and improve classification performance. Extensive experiments on four benchmarks datasets, including Indian Pines, Pavia University, Salinas, and Houston2013, yield an average overall classification accuracy of 81.65%, 89.38%, 93.47% and 83.93%, which sufficiently demonstrate that SSRNet can exceed expectations compared to state-of-the-art methods.
- 23 June 2022
- Published by MDPI AG in Remote Sensing
- Vol. 14 (13) , 2997
Open AccessSelf-Supervised Assisted Semi-Supervised Residual Network for Hyperspectral Image Classification
Due to the scarcity and high cost of labeled hyperspectral image (HSI) samples, many deep learning methods driven by massive data cannot achieve the intended expectations. Semi-supervised and self-supervised algorithms have advantages in coping with this phenomenon. This paper primarily concentrates on applying self-supervised strategies to make strides in semi-supervised HSI classification. Notably, we design an effective and a unified self-supervised assisted semi-supervised residual network (SSRNet) framework for HSI classification. The SSRNet contains two branches, i.e., a semi-supervised and a self-supervised branch. The semi-supervised branch improves performance by introducing HSI data perturbation via a spectral feature shift. The self-supervised branch characterizes two auxiliary tasks, including masked bands reconstruction and spectral order forecast, to memorize the discriminative features of HSI. SSRNet can better explore unlabeled HSI samples and improve classification performance. Extensive experiments on four benchmarks datasets, including Indian Pines, Pavia University, Salinas, and Houston2013, yield an average overall classification accuracy of 81.65%, 89.38%, 93.47% and 83.93%, which sufficiently demonstrate that SSRNet can exceed expectations compared to state-of-the-art methods.
- 24 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-13
Local Correntropy Matrix Representation for Hyperspectral Image Classification
The hyperspectral images (HSIs) classification technique has received widespread attention in the field of remote sensing. However, how to achieve satisfactory classification performance in the presence of a large amount of noise is still a problem worthy of consideration. In this article, a local correntropy matrix (LCEM)-based spatial–spectral feature representation method is proposed for HSI classification. Motivated by the successful application of information-theoretic learning (ITL), we propose to adopt correntropy matrix to represent the spatial–spectral features of HSI. Specifically, the dimension reduction is first performed on the original hyperspectral data. Then, for each pixel, we select its local neighbors within a sliding window using cosine distance for the construction of the LCEM. In this way, each pixel can be characterized as an LCEM. Finally, all the correntropy matrices are fed into a support vector machine (SVM) for final classification. In addition, we also propose a novel way to determine the size of the local window based on standard deviation. Because the LCEM as the feature descriptor can characterize discriminative spatial–spectral features, the proposed method has shown great interclass separability and intraclass compactness. Compared with other advanced approaches, the proposed LCEM method has achieved competitive performance in both evaluation indexes and visual effects, especially when the training size is very small.
- 24 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-13
Local Correntropy Matrix Representation for Hyperspectral Image Classification
The hyperspectral images (HSIs) classification technique has received widespread attention in the field of remote sensing. However, how to achieve satisfactory classification performance in the presence of a large amount of noise is still a problem worthy of consideration. In this article, a local correntropy matrix (LCEM)-based spatial–spectral feature representation method is proposed for HSI classification. Motivated by the successful application of information-theoretic learning (ITL), we propose to adopt correntropy matrix to represent the spatial–spectral features of HSI. Specifically, the dimension reduction is first performed on the original hyperspectral data. Then, for each pixel, we select its local neighbors within a sliding window using cosine distance for the construction of the LCEM. In this way, each pixel can be characterized as an LCEM. Finally, all the correntropy matrices are fed into a support vector machine (SVM) for final classification. In addition, we also propose a novel way to determine the size of the local window based on standard deviation. Because the LCEM as the feature descriptor can characterize discriminative spatial–spectral features, the proposed method has shown great interclass separability and intraclass compactness. Compared with other advanced approaches, the proposed LCEM method has achieved competitive performance in both evaluation indexes and visual effects, especially when the training size is very small.