Recognizing Human-Object Interactions in Still Images by Modeling the Mutual Context of Objects and Human Poses
Authors:
- 6 March 2012
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 34 (9) , 1691-1703
Abstract
Detecting objects in cluttered scenes and estimating articulated human body parts from 2D images are two challenging problems in computer vision. The difficulty is particularly pronounced in activities involving human-object interactions (e.g., playing tennis), where the relevant objects tend to be small or only partially visible and the human body parts are often self-occluded. We observe, however, that objects and human poses can serve as mutual context to each other-recognizing one facilitates the recognition of the other. In this paper, we propose a mutual context model to jointly model objects and human poses in human-object interaction activities. In our approach, object detection provides a strong prior for better human pose estimation, while human pose estimation improves the accuracy of detecting the objects that interact with the human. On a six-class sports data set and a 24-class people interacting with musical instruments data set, we show that our mutual context model outperforms state of the art in detecting very difficult objects and estimating human poses, as well as classifying human-object interaction activities.Keywords
This publication has 52 references indexed in Scilit:
- Modeling mutual context of object and human pose in human-object interaction activities2010
- Discriminative models for static human-object interactions2010
- Recognizing human actions from still images with latent poses2010
- Tracking people interacting with objects2010
- Poselets: Body part detectors trained using 3D human pose annotations2009
- Progressive search space reduction for human pose estimation2008
- Viewpoint-independent object class detection using 3D Feature Maps2008
- Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories2006
- Histograms of Oriented Gradients for Human Detection2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004