Two-Stage Fine-Tuning of Large Vision-Language Models with Hierarchical Prompting for Few-Shot Object Detection in Remote Sensing Images
Preprint
- 23 December 2025
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Few-shot object detection (FSOD) in high-resolution remote sensing (RS) imagery remains challenging due to scarce annotations, large intra-class variability, and high visual similarity between categories, which together limit the generalization ability of convolutional neural network (CNN)-based detectors. To address this issue, we explore leveraging large vision-language models (LVLMs) for FSOD in RS. We propose a two-stage, parameter-efficient fine-tuning framework with hierarchical prompting that adapts Qwen3-VL for object detection. In the first stage, low-rank adaptation (LoRA) modules are inserted into the vision and text encoders and trained jointly with a Detection Transformer (DETR)-style detection head on fully annotated base classes under three-level hierarchical prompts. In the second stage, the vision LoRA parameters are frozen, the text encoder is updated using K-shot novel-class samples, and the detection head is partially frozen, with selected components refined using the same three-level hierarchical prompting scheme. To preserve base-class performance and reduce class confusion, we further introduce knowledge distillation and semantic consistency losses. Experiments on the DIOR and NWPU VHR-10.v2 datasets show that the proposed method consistently improves novel-class performance while maintaining competitive base-class accuracy and surpasses existing baselines, demonstrating the effectiveness of integrating hierarchical semantic reasoning into LVLM-based FSOD for RS imagery.Keywords
All Related Versions
This publication has 61 references indexed in Scilit:
- A Lightweight Normalization-Free Architecture for Object Detection in High-Spatial-Resolution Remote Sensing ImageryIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025
- RS-TOD: Tiny object detection model in Remote Sensing ImageryRemote Sensing Applications: Society and Environment, 2025
- YOLOv8-SP: ship object detection in optical remote sensing imageryMarine Geodesy, 2025
- PR-Deformable DETR: DETR for Remote Sensing Object DetectionIEEE Geoscience and Remote Sensing Letters, 2024
- Context Feature Integration and Balanced Sampling Strategy for Small Weak Object Detection in Remote Sensing ImageryIEEE Geoscience and Remote Sensing Letters, 2024
- Generalized few-shot object detection in remote sensing imagesISPRS Journal of Photogrammetry and Remote Sensing, 2022
- Prototype-CNN for Few-Shot Object Detection in Remote Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2021
- Few-Shot Object Detection on Remote Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2021
- Object detection in optical remote sensing images: A survey and a new benchmarkISPRS Journal of Photogrammetry and Remote Sensing, 2020
- Multi-class geospatial object detection and geographic image classification based on collection of part detectorsISPRS Journal of Photogrammetry and Remote Sensing, 2014