Multiscale Random-Shape Convolution and Adaptive Graph Convolution Fusion Network for Hyperspectral Image Classification
- 18 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-17
Abstract
Convolution neural networks (CNNs) are extensively utilized in hyperspectral image (HSI) classification due to their remarkable capability to extract features from patterns with fixed shapes. These networks have been shown to effectively capture features at the pixel level. However, the fixed shape of convolution kernels poses a challenge for CNNs to adapt to the diverse shapes found in HSIs. Graph neural networks (GNNs), particularly graph convolution networks (GCNs), possess robust feature extraction capabilities on graph structures and are extensively applied in HSI classification. However, one significant challenge in using GNNs is the selection of appropriate neighboring nodes for information aggregation. To address the existing challenges of GCN and CNN and leverage their respective advantages, this article introduces a novel patch-based CNN-GCN fusion classification network, named multiscale random-shape convolution and adaptive graph convolution fusion network (MRCAGCFN). It consists of a spectral transformation module (STM) and three main modules we proposed: a multiscale random-shape convolution (RSC) module for extracting convolution features, where the shape of the convolution kernel is randomized and a multiscale approach is applied to enhance adaptability to data with diverse shapes; an adaptive feature-fusion graph convolution module (AFGCM) for extracting graph convolution features, where the weights for neighborhood aggregation are learned adaptively to reduce feature fusion from dissimilar nodes and strengthen feature fusion from similar nodes; and an adaptive local feature processing module for processing features, where two different methods are employed to convert patch-level features to pixel-level features, thereby improving feature representation. MRCAGCFN combines the strengths of CNN and GCN while introducing enhancements to better accommodate diverse feature shapes. Experimental results on three HSI classification datasets demonstrate that our proposed MRCAGCFN outperforms some existing methods. The codes of our MRCAGCFN will be available at https://github.com/shengrunhua/MRCAGCFN .Keywords
Funding Information
- National Natural Science Foundation of China (62071168)
- Natural Science Foundation of Jiangsu Province (BK20211201)
- Open Research Fund of Shaanxi Key Laboratory of Optical Remote Sensing and Intelligent Information Processing (KF20230301)
- China Postdoctoral Science Foundation (2021M690885)
This publication has 55 references indexed in Scilit:
- Temporal difference-guided network for hyperspectral image change detectionInternational Journal of Remote Sensing, 2023
- Local aggregation and global attention network for hyperspectral image classification with spectral-induced aligned superpixel segmentationExpert Systems with Applications, 2023
- Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samplesInternational Journal of Applied Earth Observation and Geoinformation, 2022
- Hyperspectral Image Classification Based on Long Short Term Memory Network2018
- Deformable Convolutional Networks2017
- Understanding of a convolutional neural network2017
- Hyperspectral Image Classification With Rotation Random Forest Via KPCAIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017
- Linear Versus Nonlinear PCA for the Classification of Hyperspectral Data Based on the Extended Morphological ProfilesIEEE Geoscience and Remote Sensing Letters, 2011
- Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles2007
- Support vector machines for classification of hyperspectral data2002