Explicit Modeling of Human-Object Interactions in Realistic Videos
- 10 August 2012
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 35 (4) , 835-848
Abstract
We introduce an approach for learning human actions as interactions between persons and objects in realistic videos. Previous work typically represents actions with low-level features such as image gradients or optical flow. In contrast, we explicitly localize in space and track over time both the object and the person, and represent an action as the trajectory of the object w.r.t. to the person position. Our approach relies on state-of-the-art techniques for human detection [32], object detection [10], and tracking [39]. We show that this results in human and object tracks of sufficient quality to model and localize human-object interactions in realistic videos. Our human-object interaction features capture the relative trajectory of the object w.r.t. the human. Experimental results on the Coffee and Cigarettes dataset [25], the video dataset of [19], and the Rochester Daily Activities dataset [29] show that 1) our explicit human-object model is an informative cue for action recognition; 2) it is complementary to traditional low-level descriptors such as 3D--HOG [23] extracted over human tracks. We show that combining our human-object interaction features with 3D-HOG improves compared to their individual performance as well as over the state of the art [23], [29].Keywords
This publication has 44 references indexed in Scilit:
- Weakly Supervised Learning of Interactions between Humans and ObjectsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2011
- Actom sequence models for efficient action detection2011
- Robust sequence alignment for actor–object interaction recognition: Discovering actor–object statesComputer Vision and Image Understanding, 2011
- Large Displacement Optical Flow: Descriptor Matching in Variational Motion EstimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Automatic annotation of human actions in video2009
- Observing Human-Object Interactions: Using Spatial and Functional Compatibility for RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Actions as Space-Time ShapesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Tracking People by Learning Their AppearanceIEEE Transactions on Pattern Analysis and Machine Intelligence, 2006
- On-line Boosting and Vision2006
- Histograms of Oriented Gradients for Human Detection2005