SARCLIP: The First Vision–Language Foundation Model for SAR Image
- 7 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-11
Abstract
Foundation models have achieved remarkable breakthroughs across various domains, with the widely use of Masked Image Modeling (MIM) and Self-Supervised Learning (SSL). However, these models lack language comprehension, which limits their ability to generalize in downstream tasks and zero-shot applications. The CLIP (Contrastive Language-Image Pretraining) model overcomes these challenges by integrating textual information and performs exceptionally in remote sensing. However, no such multimodal model exists for the SAR (Synthetic Aperture Radar) domain. To bridge this gap, we propose SARCLIP, a vision-language model specially designed for SAR, which leverages textual supervision to enhance visual representation and achieves fine-grained vision-language alignment through modality alignment. Firstly, to overcome the lack of SAR image-text datasets, we propose the Detection-to-Caption (D2C) algorithm, which transforms heterogeneous format of object detection into diverse captions, ultimately creating the vision-language dataset, MMSAR. Additionally, to address the distortion and information loss caused by downsampling in high-resolution SAR images, we propose AnyReader, a dynamic module capable of processing images of any size through a low-resolution encoder. We validate SARCLIP’s generalization and performance on five downstream tasks: image-text retrieval, zero-shot classification, few-shot classification, linear probing and object counting. Especially, we introduce a new counting benchmark, SARCount. Our results demonstrate that SARCLIP consistently outperforms the CLIP baseline across various model sizes. Impressively, SARCLIP achieved a 7.07% higher average recall on the retrieval task compared to the largest CLIP model. In zero-shot classification, SARCLIP surpasses CLIP by 7.8% in average accuracy across three downstream datasets.Keywords
Funding Information
- National Natural Science Foundation of China (62371192)
This publication has 36 references indexed in Scilit:
- RS-CLIP: Zero shot remote sensing scene classification via contrastive vision-language supervisionInternational Journal of Applied Earth Observation and Geoinformation, 2023
- Siamese Image Modeling for Self-Supervised Vision Representation Learning2023
- Understanding Masked Autoencoders via Hierarchical Latent Variable Models2023
- Understanding Masked Image Modeling via Learning Occlusion Invariant Feature2023
- Negative-Aware Attention Framework for Image-Text Matching2022
- FLAVA: A Foundational Language And Vision Alignment Model2022
- Masked Autoencoders Are Scalable Vision Learners2022
- The risks associated with Artificial General Intelligence: A systematic reviewJournal of Experimental & Theoretical Artificial Intelligence, 2021
- HRSID: A High-Resolution SAR Images Dataset for Ship Detection and Instance SegmentationIEEE Access, 2020
- FUSAR-Ship: building a high-resolution SAR-AIS matchup dataset of Gaofen-3 for ship detection and recognitionScience China Information Sciences, 2020