CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentation
- 12 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 2816-2829
Abstract
Continual video instance segmentation (CVIS) requires the plasticity to absorb new categories while maintaining the stability to retain previously learned knowledge. Crucially, the model must also preserve temporal consistency of instances across video frames. In this work, we introduce Contrastive Residual Injection and Semantic Prompting (CRISP), a framework tailored to address instance-wise, category-wise, and task-wise confusion in CVIS. For instance-wise learning, we model instance tracking and construct instance correlation loss, which emphasizes the correlation with the prior query space while strengthening the specificity of the current task query. For category-wise learning, we build an adaptive residual semantic prompt (ARSP) learning framework, which constructs a learnable semantic residual prompt pool generated by category text and uses an adjustive query-prompt matching mechanism to build a mapping relationship between the query of the current task and the semantic residual prompt. Meanwhile, a semantic consistency loss based on the contrastive learning is introduced to maintain semantic coherence between object queries and residual prompts during incremental training. For task-wise learning, to ensure the correlation at the inter-task level within the query space, we introduce a concise yet powerful initialization strategy for incremental prompts. Extensive experiments on YouTube-VIS-2019 and YouTube-VIS-2021 datasets demonstrate that CRISP significantly outperforms existing continual segmentation methods in the long-term continual video instance segmentation task, avoiding catastrophic forgetting and effectively improving segmentation and classification performance. The code is available at https://github.com/01upup10/CRISP.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2024YFB4707700)
- National Natural Science Foundation of China (U23A20343)
- National Natural Science Foundation of China (T2594604030)
- National Natural Science Foundation of China (T2594604035)
- National Natural Science Foundation of China (62303447)
- Natural Science Foundation of Liaoning Province (2025-BS-0193)
- Natural Science Foundation of Liaoning Province (2024-MSBA-81)
- Fundamental Research Project of Shenyang Institute of Automation (2024JC3K01)
- State Key Laboratory of Robotics and Intelligent Systems (2025-Z09)
This publication has 65 references indexed in Scilit:
- Semantic Residual Prompts for Continual Learning2024
- Strike a Balance in Continual Panoptic Segmentation2024
- STSP: Spatial-Temporal Subspace Projection for Video Class-Incremental Learning2024
- DualPrompt: Complementary Prompting for Rehearsal-Free Continual Learning2022
- RBC: Rectifying the Biased Context in Continual Semantic Segmentation2022
- SOLO: Segmenting Objects by Locations2020
- End-to-End Object Detection with Transformers2020
- Principal Component Analysis2011
- Principal component analysisWIREs Computational Statistics, 2010
- Lifelong Learning Algorithms1998