Abstract
Recent advancements in contrastive learning have led to significant progress in multimodal remote sensing data classification, such as hyperspectral and light detection and ranging (LiDAR) data, particularly in scenarios with limited labeled samples. However, these methods primarily rely on the contrast between positive and negative pairs, which makes them susceptible to interference from negative sample selection, thereby impairing effective multimodal fusion and ultimately affecting classification accuracy. To address this issue, this article introduces a novel framework for the joint classification of hyperspectral image (HSI) and LiDAR, termed the multipositive matching-enhanced contrastive learning (mPMCL). The proposed framework facilitates contrastive training by matching multiple positive pairs across different modalities and hierarchical features of the same object, hence effectively enhancing multimodal fusion and improving classification performance. Specifically, the hierarchical cross-modal interaction fusion (HCIF) module performs hierarchical and consistency-aware fusion by using a bidirectional cross-attention mechanism to integrate low-level, cross-modal interaction, and high-level semantic features from HSI and LiDAR data. This process gradually aligns heterogeneous representations across stages and captures complementary spectral–elevation cues, leading to more stable and discriminative multimodal features. In addition, a multipositive matching strategy (mPMS) is developed to construct multiple positive pairs by matching high-level semantic features of the same object with low-level and cross-modal fusion features from different modalities. By performing contrastive training on these positive pairs, the proposed method avoids the sensitivity of traditional approaches to negative sample selection, while further enhancing the effectiveness of multimodal information fusion. The pivotal contribution of the proposed framework lies in demonstrating the importance of positive matching enhanced contrastive learning strategies for effective multimodal information fusion. Experimental results on several publicly available datasets show that the proposed framework outperforms existing state-of-the-art methods. Our code is publicly available at https://github.com/HoppouCJ/mPMCL
Funding Information
  • Postdoctoral Research Foundation of China (2024M751474)
  • Scientific Research Foundation of Hunan Provincial Education Department (2022-KFJJ-005)
  • Project of Sihong County Active Fault Detection and Earthquake Hazard Assessment (2023h659)
  • National Natural Science Foundation of China (62505008)
  • National Natural Science Foundation of China (72403085)
  • Scientific Research Foundation of Hunan Education Department (24B0569)