Contextual Action Recognition with R*CNN
Preprint
Abstract
There are multiple cues in an image which reveal what action a person is performing. For example, a jogger has a pose which is characteristic for the action, but the scene (e.g. road, trail) and the presence of other joggers can be an additional source of information. In this work, we exploit the simple observation that actions are accompanied by contextual cues to build a strong action recognition system. We adapt RCNN to use more than one region for classification while still maintaining the ability to localize the action. We call our system R*CNN. The action-specific models and the feature maps are trained jointly, allowing for action specific representations to emerge. R*CNN achieves 89% mean AP on the PASAL VOC Action dataset, outperforming all other approaches in the field by a significant margin.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Fine-Grained Activity Recognition with Holistic and Pose Based Features2014
- Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks2014
- 2D Human Pose Estimation: New Benchmark and State of the Art Analysis2014
- Action recognition from a distributed representation of pose and appearance2011
- Combining randomization and discrimination for fine-grained image categorization2011
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- The role of context in object recognitionTrends in Cognitive Sciences, 2007
- Histograms of Oriented Gradients for Human Detection2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Scene perception: Detecting and judging objects undergoing relational violationsCognitive Psychology, 1982