On the Effectiveness of Textual Prompting With Lightweight Fine-Tuning for SAM3 Remote Sensing Segmentation
- 29 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 23 (1545598X) , 6009705
Abstract
Remote sensing (RS) segmentation is limited by scarce annotations and domain gaps between overhead and natural imagery, motivating effective adaptation under constrained supervision. SAM3’s concept-driven framework enables mask generation from textual prompts without target-specific modification. We evaluate SAM3 across four RS targets, sources, and resolutions, comparing textual, geometric, and hybrid prompting under lightweight fine-tuning at increasing supervision levels, and zero-shot inference. Results show that combining semantic and geometric cues consistently yields the best performance, while text-only prompting performs worst, particularly for irregular targets, reflecting limited semantic alignment between textual concepts and overhead appearances. Nevertheless, lightly fine-tuned textual prompting offers a favorable performance-effort trade-off for regular targets. Performance improves sharply from zero-shot to fine-tuning, followed by diminishing returns, indicating that modest annotation effort may suffice. Persistent Precision– IoU gaps further reveal under-segmentation and boundary errors as dominant failure modes, especially for irregular and less prevalent targets.Keywords
All Related Versions
Funding Information
- Israel Science Foundation (299/23)
This publication has 16 references indexed in Scilit:
- Strong and Weak Prompt Engineering for Remote Sensing Image-Text Cross-Modal RetrievalIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- Dual-Alignment CLIP: Task-Specific Alignment of Text and Visual Features for Few-Shot Remote Sensing Scene ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation ModelIEEE Transactions on Geoscience and Remote Sensing, 2024
- The Segment Anything Model (SAM) for remote sensing applications: From zero to one shotInternational Journal of Applied Earth Observation and Geoinformation, 2023
- Segment Anything2023
- Reviving Iterative Training with Mask Guidance for Interactive Segmentation2022
- Image Segmentation Using Text and Image Prompts2022
- Remote sensing image segmentation advances: A meta-analysisISPRS Journal of Photogrammetry and Remote Sensing, 2021
- ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed dataISPRS Journal of Photogrammetry and Remote Sensing, 2020