StableFusion: Continual Video Retrieval via Frame Adaptation
Preprint
Abstract
Text-to-Video Retrieval (TVR) aims to match videos with corresponding textual queries, yet the continual influx of new video content poses a significant challenge for maintaining system performance over time. In this work, we introduce the first benchmark for Continual Text-to-Video Retrieval (CTVR) to overcome these limitations. Our analysis reveals that current TVR methods based on pre-trained models struggle to retain plasticity when adapting to new tasks, while existing continual learning approaches experience catastrophic forgetting, resulting in semantic misalignment between historical queries and stored video features. To address these challenges, we propose StableFusion, a novel CTVR framework comprising two main components: the Frame Fusion Adapter (FFA), which captures temporal dynamics in video content while preserving model flexibility, and the Task-Aware Mixture-of-Experts (TAME), which maintains consistent semantic alignment between queries across tasks and the stored video features. Comprehensive evaluations on two benchmark datasets under various task settings demonstrate that StableFusion outperforms existing continual learning and TVR methods, achieving superior retrieval performance with minimal degradation on earlier tasks in the context of continuous video streams. Our code is available at: https://github.com/JasonCodeMaker/CTVRKeywords
All Related Versions
This publication has 14 references indexed in Scilit:
- FastEdit: fast text-guided single-image editing via semantic-aware diffusion fine-tuningPattern Recognition, 2025
- Learning Without Forgetting for Vision-Language ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
- Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You NeedInternational Journal of Computer Vision, 2024
- DiffusionRet: Generative Text-Video Retrieval with Diffusion Model2023
- TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval2022
- CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioningNeurocomputing, 2022
- Semantics Disentangling for Generalized Zero-Shot Learning2021
- Embracing Change: Continual Learning in Deep Neural NetworksTrends in Cognitive Sciences, 2020
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips2019
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019