Deep Sequential Context Networks for Action Prediction
Authors:
- Yu KongYKYu Kong
- Zhiqiang TaoZTZhiqiang Tao
- Yun FuYFYun Fu
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 3662-3670
Abstract
This paper proposes efficient and powerful deep networks for action prediction from partially observed videos containing temporally incomplete action executions. Different from after-the-fact action recognition, action prediction task requires action labels to be predicted from these partially observed videos. Our approach exploits abundant sequential context information to enrich the feature representations of partial videos. We reconstruct missing information in the features extracted from partial videos by learning from fully observed action videos. The amount of the information is temporally ordered for the purpose of modeling temporal orderings of action segments. Label information is also used to better separate the learned features of different categories. We develop a new learning formulation that enables efficient model training. Extensive experimental results on UCF101, Sports-1M and BIT datasets demonstrate that our approach remarkably outperforms state-of-the-art methods, and is up to 300× faster than these methods. Results also show that actions differ in their prediction characteristics, some actions can be correctly predicted even though only the beginning 10% portion of videos is observed.Keywords
This publication has 25 references indexed in Scilit:
- Interactive Phrases: Semantic Descriptionsfor Human Interaction RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Prediction of Human Activity by Discovering Temporal Sequence PatternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Sequential event predictionMachine Learning, 2013
- Poselet Key-Framing: A Model for Human Activity Recognition2013
- Spatiotemporal Deformable Part Models for Action Detection2013
- Dense Trajectories and Motion Boundary Descriptors for Action RecognitionInternational Journal of Computer Vision, 2013
- Complex Events Detection Using Data-Driven Concepts2012
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Behavior Recognition via Sparse Spatio-Temporal Features2006
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005