Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images
Authors:
- Shanwen WangSWShanwen Wang
- Xin SunXSXin Sun
- Danfeng HongDHDanfeng Hong
- Fei ZhouFZFei Zhou
Preprint
Abstract
The semi-supervised semantic segmentation (S4) can learn rich visual knowledge from low-cost unlabeled images. However, traditional S4 architectures all face the challenge of low-quality pseudo-labels, especially for the teacher-student framework.We propose a novel SemiEarth model that introduces vision-language models (VLMs) to address the S4 issues for the remote sensing (RS) domain. Specifically, we invent a VLM pseudo-label purifying (VLM-PP) structure to purify the teacher network's pseudo-labels, achieving substantial improvements. Especially in multi-class boundary regions of RS images, the VLM-PP module can significantly improve the quality of pseudo-labels generated by the teacher, thereby correctly guiding the student model's learning. Moreover, since VLM-PP equips VLMs with open-world capabilities and is independent of the S4 architecture, it can correct mispredicted categories in low-confidence pseudo-labels whenever a discrepancy arises between its prediction and the pseudo-label. We conducted extensive experiments on multiple RS datasets, which demonstrate that our SemiEarth achieves SOTA performance. More importantly, unlike previous SOTA RS S4 methods, our model not only achieves excellent performance but also offers good interpretability. The code is released at https://github.com/wangshanwen001/SemiEarth.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Domain generalization for semantic segmentation of remote sensing images via vision foundation model fine-tuningISPRS Journal of Photogrammetry and Remote Sensing, 2025
- PAMSNet: A point annotation-driven multi-source network for remote sensing semantic segmentationISPRS Journal of Photogrammetry and Remote Sensing, 2025
- RSProtoSemiSeg: Semi-supervised semantic segmentation of high spatial resolution remote sensing images with probabilistic distribution prototypesISPRS Journal of Photogrammetry and Remote Sensing, 2025
- RSGPT: A remote sensing vision language model and benchmarkISPRS Journal of Photogrammetry and Remote Sensing, 2025
- SkyEyeGPT: Unifying remote sensing vision-language tasks via instruction tuning with large language modelISPRS Journal of Photogrammetry and Remote Sensing, 2025
- Decouple and weight semi-supervised semantic segmentation of remote sensing imagesISPRS Journal of Photogrammetry and Remote Sensing, 2024
- ChangeCLIP: Remote sensing change detection with multimodal vision-language representation learningISPRS Journal of Photogrammetry and Remote Sensing, 2024
- SemiRoadExNet: A semi-supervised network for road extraction from remote sensing imagery via adversarial learningISPRS Journal of Photogrammetry and Remote Sensing, 2023
- Semi-supervised bidirectional alignment for Remote Sensing cross-domain scene classificationISPRS Journal of Photogrammetry and Remote Sensing, 2022
- Meta captioning: A meta learning based remote sensing image captioning frameworkISPRS Journal of Photogrammetry and Remote Sensing, 2022