Bo Peng

Video class-incremental learning (VCIL) aims to progressively recognize novel action categories while preserving spatial–temporal knowledge of previous tasks. Unlike image class-incremental learning (CIL), VCIL requires simultaneously capturing spatial semantics and temporal dynamics, which makes catastrophic forgetting more significant. Prompt-based learning has recently made remarkable advancements in VCIL. However, they primarily optimize a predefined static prompt for all action sequences in a group-level manner, which overlooks the diverse spatial–temporal characteristics across frames and exhibits limited generalization capability for future action categories. To address the above limitations, we propose a novel exemplar-free VCIL framework termed STCP-low-rank dynamic routing (LRDR) that consists of two crucial innovations, i.e., the spatiotemporal context-aware prompting (SCAP) and LRDR. The former dynamically generates instance-level prompts based on the input video. Specifically, the frame-level prompt is developed to adaptively emphasize fine-grained details in each frame by leveraging the attention-guided spatial activation module. Meanwhile, we also design the cross-frame prompt to capture the differential importance of sequences, allowing the model to focus on key frames and facilitate its ability to explore temporal dependencies. Furthermore, we introduce a parameter-efficient LRDR that achieves dynamic scalability by incorporating mixture-of-experts low-rank spatial–temporal adapters, which can maintain old prompt knowledge and cross-task collaboration as the trainable SCAP also suffers from catastrophic forgetting when learning from incremental tasks. Finally, we propose a novel prompt correction mechanism (PCM) that prevents the proposed SCAP from acquiring ineffective class-wise spatiotemporal representations via the discriminative class-level prompt knowledge. Extensive experiments are conducted on four video benchmarks, and our approach consistently achieved substantial gains over state-of-the-art methods in VCIL.

Recent advancements in weakly supervised semantic segmentation (WSSS) have shown promise by using the contrastive language-image pretraining (CLIP) model to generate pseudo-labels. However, directly applying the CLIP model without considering interclass relationships in downstream tasks has resulted in suboptimal transferability and generalization. To address these challenges, we propose CLIP graph adapter (CLIP-GA), a novel approach that integrates both textual and visual structural knowledge to generate high-quality initial class activation maps (CAMs) for each object class. Our method introduces a dual-graph adaptive strategy, comprising a textual subgraph and a visual subgraph and employs cross-modal graph attention (CGA) for effective fusion. The framework includes three specialized loss functions that help to capture more complete object regions while minimizing the activation of background areas closely related to foreground objects. In addition, we implement the superpixel consistency to refine pseudo-labels and introduce a graph reasoning attention (GRA) module to build global contextual relationships within visual features for the segmentation network. Extensive experiments on the PASCAL VOC 2012 and MS COCO 2014 datasets have convincingly demonstrated the effectiveness of CLIP-GA compared with other state-of-the-art methods. Our code is provided at: https://github.com/JIA-ZHANG666/CLIP-GA