From Actemes to Action: A Strongly-Supervised Representation for Detailed Action Understanding
Authors:
- Weiyu ZhangWZWeiyu Zhang
- Menglong ZhuMZMenglong Zhu
- Konstantinos G. DerpanisKDKonstantinos G. Derpanis
- 1 December 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15505499,p. 2248-2255
Abstract
This paper presents a novel approach for analyzing human actions in non-scripted, unconstrained video settings based on volumetric, x-y-t, patch classifiers, termed actemes. Unlike previous action-related work, the discovery of patch classifiers is posed as a strongly-supervised process. Specifically, key point labels (e.g., position) across space time are used in a data-driven training process to discover patches that are highly clustered in the space time key point configuration space. To support this process, a new human action dataset consisting of challenging consumer videos is introduced, where notably the action label, the 2D position of a set of key points and their visibilities are provided for each video frame. On a novel input video, each acteme is used in a sliding volume scheme to yield a set of sparse, non-overlapping detections. These detections provide the intermediate substrate for segmenting out the action. For action classification, the proposed representation shows significant improvement over state-of-the-art low-level features, while providing spatiotemporal localization as additional output, which sheds further light into detailed action understanding.Keywords
This publication has 26 references indexed in Scilit:
- Recognizing 50 human action categories of web videosMachine Vision and Applications, 2012
- Articulated pose estimation with flexible mixtures-of-parts2011
- Evaluation of local spatio-temporal features for action recognition2009
- A Spatio-Temporal Descriptor Based on 3D-Gradients2008
- Event Detection in Crowded Videos2007
- Using Multiple Segmentations to Discover Objects and their Extent in Image Collections2006
- Behavior Recognition via Sparse Spatio-Temporal Features2006
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005
- Recognizing human actions: a local SVM approach2004
- Facial Attention and Spacetime FragmentsAxiomathes, 2003