CLIP-Driven with Dynamic Feature Selection and Alignment Network for Referring Remote Sensing Image Segmentation
Preprint
- 17 September 2025
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Referring Remote Sensing Image Segmentation (RRSIS) aims to accurately locate and segment target objects in high-resolution aerial imagery based on natural language descriptions. The current RRSIS model faces numerous gaps due to the significant differences between remote sensing images and natural images, including scale variations, object rotation, and the difficulty of matching complex linguistic queries with spatially variable targets. Existing methods often rely on high-level semantic features or multi-stage cross-modal alignment, resulting in long training times and inefficiencies with complex queries. In this context, we propose CLIP-Driven with Dynamic Feature Selection and Alignment Network(CD2FSAN) , a novel framework that includes information-theoretic feature selection, adaptive multi-scale aggregation and alignment , and dynamic rotation correction decoder to better align remote sensing visual features with textual descriptions. Specially CD2FSAN dynamically selects visual features that best match the language description based on the principle of maximizing cross-modal information, alleviating domain shift from the pretraining of CLIP on natural images, and integrates language information during encoding. The framework also incorporates a multi-scale feature aggregation and alignment mechanism, ensuring precise cross-modal alignment, particularly for small targets. Additionally, CD2FSAN introduces a differentiable affine transformation-based dynamic rotation correction mechanism, enabling the network to adaptively adjust object orientations, improving segmentation accuracy. Experiments on three standard datasets, RefSegRS, RRSIS-D, and RISBench, demonstrate CD2FSAN’s superior performance in terms of oIoU, mIoU, and precision. Ablation studies and qualitative visualizations validate the efficacy of each module, confirming the framework’s robustness in handling spatial variations, rotation, and cross-modal alignment, significantly reducing the cross modality gap in CLIP-based single-stage RRSIS tasks.Keywords
All Related Versions
- Version 1, 2025-09-17, Preprints
- Published version: Remote Sensing, 17 (22), 3675.
This publication has 27 references indexed in Scilit:
- PatchOut: A novel patch-free approach based on a transformer-CNN hybrid framework for fine-grained land-cover classification on large-scale airborne hyperspectral imagesInternational Journal of Applied Earth Observation and Geoinformation, 2025
- A fast hybrid approach for continuous land cover change monitoring and semantic segmentation using satellite time seriesInternational Journal of Applied Earth Observation and Geoinformation, 2024
- Methods and datasets on semantic segmentation for Unmanned Aerial Vehicle remote sensing images: A reviewISPRS Journal of Photogrammetry and Remote Sensing, 2024
- A review of deep learning methods for semantic segmentation of remote sensing imageryExpert Systems with Applications, 2021
- Bi-Directional Relationship Inferring Network for Referring Image Segmentation2020
- Remote sensing for agricultural applications: A meta-reviewRemote Sensing of Environment, 2020
- Cross-Modal Self-Attention Network for Referring Image Segmentation2019
- Referring Image Segmentation via Recurrent Refinement Networks2018
- A survey on object detection in optical remote sensing imagesISPRS Journal of Photogrammetry and Remote Sensing, 2016
- Generation and Comprehension of Unambiguous Object Descriptions2016