Cross-Semantic Heterogeneous Modeling Network for Hyperspectral Image Classification
- 11 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-16
Abstract
The adequate and finer spectral information in hyperspectral images (HSIs) are benefit for various downstream applications like smart agriculture and environmental monitoring. In HSI classification, dual-stream convolutional networks have gained much attention and have been widely used. In patch-based hyperspectral classification tasks, however, merely using center-labeled patches could lead to an increased unlabeled noise in the data. Moreover, in the application of dual-stream network structures, heterogeneity existed in both the data and feature semantic levels to capture more representative features. To tackle these challenges, we have devised a framework called cross-semantic heterogeneous modeling network (CreatingNet), which aligns more closely with the design principles of dual-stream networks by adjusting the input size. This framework introduces a distance metric attention mechanism (DMAM) based on spectral and spatial distances to strengthen the influence of the center pixel on the entire patch. Additionally, we present a fusion module named CrossViT, which combines features with diverse structures and characteristics, leveraging their complementarity. The proposed multiscale heterogeneous fusion module allows for more effective integration of spatial and spectral features in the images. Extensive experiments on four well-known HSI datasets (Indian Pines, Pavia University, Salinas, and Houston 2013) demonstrate the superior classification performance of the proposed CreatingNet to several state-of-the-art methods. The effectiveness of the proposed model is further validated through ablation studies.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFB3900502)
This publication has 51 references indexed in Scilit:
- Deep hybrid: Multi-graph neural network collaboration for hyperspectral image classificationDefence Technology, 2023
- HyperViTGAN: Semisupervised Generative Adversarial Network With Transformer for Hyperspectral Image ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022
- Hybrid Dense Network With Attention Mechanism for Hyperspectral Image ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022
- Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future ProspectsIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021
- HResNetAM: Hierarchical Residual Network With Attention Mechanism for Hyperspectral Image ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021
- Combining t-Distributed Stochastic Neighbor Embedding With Convolutional Neural Networks for Hyperspectral Image ClassificationIEEE Geoscience and Remote Sensing Letters, 2019
- HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image ClassificationIEEE Geoscience and Remote Sensing Letters, 2019
- Locality-Preserving Discriminant Analysis in Kernel-Induced Feature Spaces for Hyperspectral Image ClassificationIEEE Geoscience and Remote Sensing Letters, 2011
- A Robust Built-Up Area Presence Index by Anisotropic Rotation-Invariant Textural MeasureIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2008
- Real-time constrained linear discriminant analysis to target detection and classification in hyperspectral imageryPattern Recognition, 2003