Direct Segment Anything Model-Remote Sensing: A Vision-language Foundation Model for Semantic Edge Detection in Remote Sensing Imagery
Author:
- 10 July 2025
- journal article
- Published by SAGE Publications in Web Intelligence
- Vol. 23 (4) , 529-542
Abstract
The Direct Segment Anything Model (DirectSAM), pre-trained on the SA-1B dataset, which serves as the training set for the Segment Anything Model, demonstrates exceptional performance in class-agnostic edge detection. This research explores its application to remote sensing imagery, emphasizing the practical significance of semantic edge detection, including structures like buildings, roadways, and coastlines. Currently, these applications are managed by separately training specialized models on individual datasets in each specific domain. We present DirectSAM-remote sensing (RS), a foundation model built upon DirectSAM. It retains the powerful segmentation abilities acquired from natural images while leveraging a large-scale dataset designed for semantic edge detection remote sensing. The dataset contains over 34k image-text-edge triplets, making it over 30 times larger than any individual dataset. DirectSAM-RS incorporates a prompter module, consisting of a text encoder and cross-attention layers added to the DirectSAM architecture, enabling flexible conditioning on target class labels or reference expressions. We assess DirectSAM-RS in both zero-shot and fine-tuning scenarios, showing that it delivers state-of-the-art results on various downstream benchmarks.Keywords
This publication has 28 references indexed in Scilit:
- A novel sea-land segmentation network for enhanced coastline extraction using satellite remote sensing imagesAdvances In Space Research, 2024
- Dense extreme inception network for edge detectionPattern Recognition, 2023
- VLP: A Survey on Vision-language Pre-trainingMachine Intelligence Research, 2023
- Hugging Face2022
- LDC: Lightweight Dense CNN for Edge DetectionIEEE Access, 2022
- Segmentation from Natural Language Expressions2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- ImageNet: A large-scale hierarchical image database2009
- Topological structural analysis of digitized binary images by border followingComputer Vision, Graphics, and Image Processing, 1985