SVAC: Scaling Is All You Need For Referring Video Object Segmentation
Preprint
Abstract
Referring Video Object Segmentation (RVOS) aims to segment target objects in video sequences based on natural language descriptions. While recent advances in Multi-modal Large Language Models (MLLMs) have improved RVOS performance through enhanced text-video understanding, several challenges remain, including insufficient exploitation of MLLMs' prior knowledge, prohibitive computational and memory costs for long-duration videos, and inadequate handling of complex temporal dynamics. In this work, we propose SVAC, a unified model that improves RVOS by scaling up input frames and segmentation tokens to enhance video-language interaction and segmentation precision. To address the resulting computational challenges, SVAC incorporates the Anchor-Based Spatio-Temporal Compression (ASTC) module to compress visual tokens while preserving essential spatio-temporal structure. Moreover, the Clip-Specific Allocation (CSA) strategy is introduced to better handle dynamic object behaviors across video clips. Experimental results demonstrate that SVAC achieves state-of-the-art performance on multiple RVOS benchmarks with competitive efficiency. Our code is available at https://github.com/lizhang1998/SVAC.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- TokenPacker: Efficient Visual Projector for Multimodal LLMInternational Journal of Computer Vision, 2025
- VideoGLaMM : A Large Multimodal Model for Pixel-Level Visual Grounding in Videos2025
- VisionZip: Longer is Better but Not Necessary in Vision Language Models2025
- VISA: Reasoning Video Object Segmentation via Large Language Models2024
- LongVLM: Efficient Long Video Understanding via Large Language Models2024
- LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models2024
- Universal Instance Perception as Object Discovery and Retrieval2023
- GRES: Generalized Referring Expression Segmentation2023
- Language as Queries for Referring Video Object Segmentation2022
- MAttNet: Modular Attention Network for Referring Expression Comprehension2018