Compressed Video Action Recognition
Authors:
- Chao-Yuan WuCWChao-Yuan Wu
- Manzil ZaheerMZManzil Zaheer
- Hexiang HuHHHexiang Hu
- R. ManmathaRMR. Manmatha
- Alexander J. SmolaASAlexander J. Smola
- Philipp KrahenbuhlPKPhilipp Krahenbuhl
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6026-6035
Abstract
Training robust deep video representations has proven to be much more challenging than learning deep image representations. This is in part due to the enormous size of raw video streams and the high temporal redundancy; the true and interesting signal is often drowned in too much irrelevant data. Motivated by that the superfluous information can be reduced by up to two orders of magnitude by video compression (using H.264, HEVC, etc.), we propose to train a deep network directly on the compressed video. This representation has a higher information density, and we found the training to be easier. In addition, the signals in a compressed video provide free, albeit noisy, motion information. We propose novel techniques to use them effectively. Our approach is about 4.6 times faster than Res3D and 2.7 times faster than ResNet-152. On the task of action recognition, our approach outperforms all the other methods on the UCF-101, HMDB-51 and Charades dataset.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Action Recognition with Improved Trajectories2013
- Dense Trajectories and Motion Boundary Descriptors for Action RecognitionInternational Journal of Computer Vision, 2013
- Evaluation of local spatio-temporal features for action recognition2009
- Detailed Real-Time Urban 3D Reconstruction from VideoInternational Journal of Computer Vision, 2007
- A Duality Based Approach for Realtime TV-L 1 Optical Flow2007
- Histograms of Oriented Gradients for Human Detection2005
- Moving object detection in wavelet compressed videoSignal Processing: Image Communication, 2005
- Fast object detection and segmentation in MPEG compressed domain2002
- MPEGCommunications of the ACM, 1991