A Key Volume Mining Deep Framework for Action Recognition
Authors:
- Wangjiang ZhuWZWangjiang Zhu
- Jie HuJHJie Hu
- Gang SunGSGang Sun
- Xudong CaoXCXudong Cao
- Yu QiaoYQYu Qiao
- 1 June 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1991-1999
Abstract
Recently, deep learning approaches have demonstrated remarkable progresses for action recognition in videos. Most existing deep frameworks equally treat every volume i.e. spatial-temporal video clip, and directly assign a video label to all volumes sampled from it. However, within a video, discriminative actions may occur sparsely in a few key volumes, and most other volumes are irrelevant to the labeled action category. Training with a large proportion of irrelevant volumes will hurt performance. To address this issue, we propose a key volume mining deep framework to identify key volumes and conduct classification simultaneously. Specifically, our framework is trained is optimized in an alternative way integrated to the forward and backward stages of Stochastic Gradient Descent (SGD). In the forward pass, our network mines key volumes for each action class. In the backward pass, it updates network parameters with the help of these mined key volumes. In addition, we propose "Stochastic out" to model key volumes from multi-modalities, and an effective yet simple "unsupervised key volume proposal" method for high quality volume sampling. Our experiments show that action recognition performance can be significantly improved by mining key volumes, and we achieve state-of-the-art performance on HMDB51 and UCF101 (93.1%).Keywords
This publication has 17 references indexed in Scilit:
- Is object localization for free? - Weakly-supervised learning with convolutional neural networks2015
- Action recognition with trajectory-pooled deep-convolutional descriptors2015
- Learning to segment moving objects in videos2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- Action Recognition with Improved Trajectories2013
- Learning Discriminative Space–Time Action Parts from Weakly Labelled VideosInternational Journal of Computer Vision, 2013
- Multiple instance classification: Review, taxonomy and comparative studyArtificial Intelligence, 2013
- Dense Trajectories and Motion Boundary Descriptors for Action RecognitionInternational Journal of Computer Vision, 2013
- A survey on vision-based human action recognitionImage and Vision Computing, 2010
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005