SASR: Sensor-Agnostic Semantic Representation Unification for Cross-Modal RGB and Hyperspectral Aerial Scene Recognition
Open Access
- 6 May 2026
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 18 (9) , 1444
Abstract
Aerial scene recognition has progressed substantially with deep learning methods for RGB and hyperspectral imagery; however, existing approaches typically operate on single modalities or rely on explicit multimodal fusion, limiting scalability, flexibility, and deployment in heterogeneous sensing environments. To address this limitation, we propose a sensor-agnostic semantic representation learning framework that formulates multimodal learning as the unification of semantic representations rather than feature-level fusion. The proposed architecture employs modality-specific encoders and projection heads to map spatial and spectral–spatial features into a shared semantic embedding space, enabling modality-invariant representation learning while preserving discriminative characteristics of each sensing modality. A composite objective integrating cross-spectral alignment, intra-class compactness regularization, and prototype-based semantic anchoring is introduced to enforce consistent embedding geometry and improve class separability across modalities. A unified classifier operating within this shared space enables reliable inference from a single modality input without requiring paired data or explicit fusion. Extensive evaluations on multiple benchmark datasets, including Houston 2013 for cross-modality RGB–hyperspectral analysis, UC Merced for independent RGB aerial scene classification, and Indian Pines for hyperspectral land-cover recognition, demonstrate the robustness and generalization capability of the proposed framework. In Houston 2013, the method achieves 96.4% (RGB) and 97.3% (hyperspectral) overall accuracy, with cross-modality transfer performance of 87.2% (RGB → HSI) and 88.7% (HSI → RGB), further improving to 97.0% and 97.8% under joint training. On UC Merced and Indian Pines, the model attains 98.7% and 97.6% overall accuracy, respectively. These results establish semantic representation unification as a scalable and effective alternative to conventional multimodal fusion for heterogeneous remote sensing environments.Keywords
Funding Information
- Princess Nourah bint Abdulrahman University Researchers (PNURSP2026R349)
- National Research Foundation of Korea (RS-2023-00218176)
- Deanship of Research and Graduate Studies at King Khalid University (RGP2/603/45)
This publication has 30 references indexed in Scilit:
- LESFuse: A lightweight dual-domain collaborative framework for high-fidelity visible-infrared image fusionApplied Soft Computing, 2026
- Modal-aware contrastive learning for hyperspectral and LiDAR classificationImage and Vision Computing, 2025
- Conditional variational underwater image enhancement with kernel decomposition and adaptive hybrid normalizationNeurocomputing, 2025
- RSGPT: A remote sensing vision language model and benchmarkISPRS Journal of Photogrammetry and Remote Sensing, 2025
- Lightweight deep learning models for aerial scene classification: A comprehensive surveyEngineering Applications of Artificial Intelligence, 2024
- Geospatial foundation models for image analysis: evaluating and enhancing NASA-IBM Prithvi’s domain adaptabilityInternational Journal of Geographical Information Science, 2024
- HSIMAE: A Unified Masked Autoencoder With Large-Scale Pretraining for Hyperspectral Image ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- A Billion-Scale Foundation Model for Remote Sensing ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding2023
- Brain-Inspired Remote Sensing Interpretation: A Comprehensive SurveyIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023