Understanding Collective Activitiesof People from Videos
Authors:
- 1 June 2014
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 36 (6) , 1242-1257
Abstract
This paper presents a principled framework for analyzing collective activities at different levels of semantic granularity from videos. Our framework is capable of jointly tracking multiple individuals, recognizing activities performed by individuals in isolation (i.e., atomic activities such as walking or standing), recognizing the interactions between pairs of individuals (i.e., interaction activities) as well as understanding the activities of group of individuals (i.e., collective activities). A key property of our work is that it can coherently combine bottom-up information stemming from detections or fragments of tracks (or tracklets) with top-down evidence. Top-down evidence is provided by a newly proposed descriptor that captures the coherent behavior of groups of individuals in a spatial-temporal neighborhood of the sequence. Top-down evidence provides contextual information for establishing accurate associations between detections or tracklets across frames and, thus, for obtaining more robust tracking results. Bottom-up evidence percolates upwards so as to automatically infer collective activity labels. Experimental results on two challenging data sets demonstrate our theoretical claims and indicate that our model achieves enhances tracking results and the best collective classification results to date.Keywords
This publication has 40 references indexed in Scilit:
- Stochastic Representation and Recognition of High-Level Group ActivitiesInternational Journal of Computer Vision, 2010
- Cutting-plane training of structural SVMsMachine Learning, 2009
- Putting Objects in PerspectiveInternational Journal of Computer Vision, 2008
- Unsupervised Learning of Human Action Categories Using Spatial-Temporal WordsInternational Journal of Computer Vision, 2008
- Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part DetectorsInternational Journal of Computer Vision, 2007
- Efficient Belief Propagation for Early VisionInternational Journal of Computer Vision, 2006
- Shape matching and object recognition using shape contextsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Event detection and analysis from video streamsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Recognizing Planned, Multiperson ActionComputer Vision and Image Understanding, 2001
- An Automatic Method of Solving Discrete Programming ProblemsEconometrica, 1960