Few-Shot Video Classification via Temporal Alignment
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10615-10624
Abstract
Difficulty in collecting and annotating large-scale video data raises a growing interest in learning models which can recognize novel classes with only a few training examples. In this paper, we propose the Ordered Temporal Alignment Module (OTAM), a novel few-shot learning framework that can learn to classify a previously unseen video. While most previous work neglects long-term temporal ordering information, our proposed model explicitly leverages the temporal ordering information in video data through ordered temporal alignment. This leads to strong data-efficiency for few-shot learning. In concrete, our proposed pipeline learns a deep distance measurement of the query video with respect to novel class proxies over its alignment path. We adopt an episode-based training scheme and directly optimize the few-shot learning objective. We evaluate OTAM on two challenging real-world datasets, Kinetics and Something-Something-V2, and show that our model leads to significant improvement of few-shot video classification over a wide range of competitive baselines and outperforms state-of-the-art benchmarks by a large margin.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Low-Shot Visual Recognition by Shrinking and Hallucinating Features2017
- The “Something Something” Video Database for Learning and Evaluating Visual Common Sense2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- Action Recognition with Improved Trajectories2013
- A Spatio-Temporal Descriptor Based on 3D-Gradients2008
- A 3-dimensional sift descriptor and its application to action recognition2007
- Dynamic Time Warping2007
- Gapped BLAST and PSI-BLAST: a new generation of protein database search programsNucleic Acids Research, 1997