Grouplet: A structured image representation for recognizing human and object interactions
Authors:
- 1 June 2010
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 9-16
Abstract
Psychologists have proposed that many human-object interaction activities form unique classes of scenes. Recognizing these scenes is important for many social functions. To enable a computer to do this is however a challenging task. Take people-playing-musical-instrument (PPMI) as an example; to distinguish a person playing violin from a person just holding a violin requires subtle distinction of characteristic image features and feature arrangements that differentiate these two scenes. Most of the existing image representation methods are either too coarse (e.g. BoW) or too sparse (e.g. constellation models) for performing this task. In this paper, we propose a new image feature representation called “grouplet”. The grouplet captures the structured information of an image by encoding a number of discriminative visual features and their spatial configurations. Using a dataset of 7 different PPMI activities, we show that grouplets are more effective in classifying and detecting human-object interactions than other state-of-the-art methods. In particular, our method can make a robust distinction between humans playing the instruments and humans co-occurring with the instruments without playing.Keywords
This publication has 22 references indexed in Scilit:
- On feature combination for multiclass object classification2009
- Unsupervised feature optimization (UFO): Simultaneous selection of multiple features with their detection parameters2009
- Observing Human-Object Interactions: Using Spatial and Functional Compatibility for RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Mirror neuron system: basic findings and clinical applicationsAnnals of Neurology, 2007
- Objects in Action: An Approach for Combining Action Understanding and Object Perception2007
- High-Performance Rotation Invariant Multiview Face DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition2006
- Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories2006
- The pyramid match kernel: discriminative classification with sets of image features2005
- Object recognition from local scale-invariant features1999