DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency
- 22 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (4) , 4642-4656
Abstract
In-context segmentation, also known as one-shot segmentation, aims to segment objects based on a single labeled example. While the Segment Anything Model (SAM) excels in interactive segmentation, it is not inherently designed for in-context tasks. To bridge this gap, we propose a new Dual Consistency SAM (DC-SAM), a prompt-tuning framework that adapts SAM and SAM2 for both image and video in-context segmentation. Instead of relying solely on pre-trained backbones, DC-SAM enhances the prompt encoder by generating high-quality visual prompts through feature fusion. Furthermore, we introduce a novel cycle-consistent cross-attention mechanism to enforce alignment between fused features and visual prompts, complemented by a dual-branch design incorporating discriminative positive and negative prompts. Additionally, we extend DC-SAM to the video domain via a novel mask-tube training strategy. To facilitate research, we curate the first In-Context Video Object Segmentation (IC-VOS) benchmark. Extensive experiments demonstrate that DC-SAM achieves state-of-the-art performance, yielding 55.5 mIoU (+1.4) on COCO-20$^{i}$ , 73.0 (+1.1) mIoU on PASCAL-5$^{i}$ , and a $\mathcal {J} { \& amp;} \mathcal{F}$ score of 71.52 on IC-VOS.
Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62572072)
- National Natural Science Foundation of China (U24B20176)
- Natural Science Foundation of Beijing Municipality (L243027)
This publication has 57 references indexed in Scilit:
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection2024
- Open-Vocabulary SAM: Segment and Recognize Twenty-Thousand Classes Interactively2024
- Self-support Few-Shot Semantic Segmentation2022
- Dense Gaussian Processes for Few-Shot Segmentation2022
- Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation2022
- Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation2022
- HM: Hybrid Masking for Few-Shot Segmentation2022
- Prototype Mixture Models for Few-Shot Semantic Segmentation2020
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018