CCFormer: Cross-Modal Cross-Attention Transformer for Classification of Hyperspectral and LiDAR Data
Open Access
Abstract
The fusion of multi-source remote sensing data has emerged as a critical technical approach to enhancing the accuracy of ground object classification. The synergistic integration of hyperspectral images and light detection and ranging data can significantly improve the capability of identifying ground objects in complex environments. However, modeling the correlation between their heterogeneous features remains a key technical challenge. Conventional methods often result in feature redundancy due to simple concatenation, making it difficult to effectively exploit the complementary information across modalities. To address this issue, this paper proposes a cross-modal cross-attention Transformer network for the classification of hyperspectral images combined with light detection and ranging data. The proposed method aims to effectively integrate the complementary characteristics of hyperspectral images and light detection and ranging data. Specifically, it employs a two-level pyramid architecture to extract multi-scale features at the shallow level, thereby overcoming the redundancy limitations associated with traditional stacking-based fusion approaches. Furthermore, an innovative cross-attention mechanism is introduced within the Transformer encoder to dynamically capture the semantic correlations between the spectral features of hyperspectral images and the elevation information from light detection and ranging data. This enables effective feature alignment and enhancement through the adaptive allocation of attention weights. Extensive experiments conducted on three publicly available datasets demonstrate that the proposed method exhibits notable advantages over existing state-of-the-art approaches.Keywords
Funding Information
- the Science and Technology Research Project of Henan Province (252102241019, 26A510005, 25B510005, 2024SJGLX0951)
- Key Scientific Research Projects of Colleges and Universities of Henan Province (252102241019, 26A510005, 25B510005, 2024SJGLX0951)
- the Higher Education Teaching Reform Research and Practice Projects of Henan Province (252102241019, 26A510005, 25B510005, 2024SJGLX0951)
This publication has 34 references indexed in Scilit:
- Multi-scale hierarchical cross fusion network for hyperspectral image and LiDAR classificationJournal of the Franklin Institute, 2025
- PyionNet: Pyramid Progressive Cross-Fusion Network for Joint Classification of Hyperspectral and LiDAR DataIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- MCFTNet: Multimodal Cross-Layer Fusion Transformer Network for Hyperspectral and LiDAR Data ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- Onboard Processing of Hyperspectral Imagery: Deep Learning Advancements, Methodologies, Challenges, and Emerging TrendsIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- UAV Hyperspectral Remote Sensing Image Classification: A Systematic ReviewIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- BIHAF-Net: Bilateral Interactive Hierarchical Adaptive Fusion Network for Collaborative Classification of Hyperspectral and LiDAR DataIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- Modality Fusion Vision Transformer for Hyperspectral and LiDAR Data Collaborative ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- Multiscale and Multidirection Feature Extraction Network for Hyperspectral and LiDAR ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- ResFPN: Residual Skip Connections in Multi-Resolution Feature Pyramid Networks for Accurate Dense Pixel Matching2021
- Land-cover classification using both hyperspectral and LiDAR dataInternational Journal of Image and Data Fusion, 2015