Few-Shot Video Classification via Representation Fusion and Promotion Learning
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 19254-19263
Abstract
Recent few-shot video classification (FSVC) works achieve promising performance by capturing similarity across support and query samples with different temporal alignment strategies or learning discriminative features via Transformer block within each episode. However, they ignore two important issues: a) It is difficult to capture rich intrinsic action semantics from a limited number of support instances within each task. b) Redundant or irrelevant frames in videos easily weaken the positive influence of discriminative frames. To address these two issues, this paper proposes a novel Representation Fusion and Promotion Learning (RFPL) mechanism with two sub-modules: meta-action learning (MAL) and reinforced image representation (RIR). Concretely, during training stage, we perform online learning for seeking a task-shared meta-action bank to enrich task-specific action representation by injecting global knowledge. Besides, we exploit reinforcement learning to obtain the importance of each frame and refine the representation. This operation maximizes the contribution of discriminative frames to further capture the similarity of support and query samples from the same category. Our RFPL framework is highly flexible that it can be integrated with many existing FSVC methods. Extensive experiments show that RFPL significantly enhances the performance of existing FSVC models when integrated with them.Keywords
This publication has 39 references indexed in Scilit:
- Incomplete Multi-view Domain Adaptation via Channel Enhancement and Knowledge Transfer2022
- Few-Shot Action Recognition with Permutation-Invariant Attention2020
- How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization2018
- Compound Memory Networks for Few-Shot Video Classification2018
- Temporal Relational Reasoning in Videos2018
- Human-level control through deep reinforcement learningNature, 2015
- One Shot Similarity Metric Learning for Action Recognition2011
- Dynamic Time Warping2007
- A new Hausdorff distance for image matchingPattern Recognition Letters, 2005
- Q-learningMachine Learning, 1992