Unsupervised learning of event AND-OR grammar and semantics from video
Authors:
- 1 November 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15505499,p. 41-48
Abstract
We study the problem of automatically learning event AND-OR grammar from videos of a certain environment, e.g. an office where students conduct daily activities. We propose to learn the event grammar under the information projection and minimum description length principles in a coherent probabilistic framework, without manual supervision about what events happen and when they happen. Firstly a predefined set of unary and binary relations are detected for each video frame: e.g. agent's position, pose and interaction with environment. Then their co-occurrences are clustered into a dictionary of simple and transient atomic actions. Recursively these actions are grouped into longer and complexer events, resulting in a stochastic event grammar. By modeling time constraints of successive events, the learned grammar becomes context-sensitive. We introduce a new dataset of surveillance-style video in office, and present a prototype system for video analysis integrating bottom-up detection, grammatical learning and parsing. On this dataset, the learning algorithm is able to automatically discover important events and construct a stochastic grammar, which can be used to accurately parse newly observed video. The learned grammar can be used as a prior to improve the noisy bottom-up detection of atomic actions. It can also be used to infer semantics of the scene. In general, the event grammar is an efficient way for common knowledge acquisition from video.Keywords
This publication has 11 references indexed in Scilit:
- Parsing video events with goal inference and intent prediction2011
- CO3 for ultra-fast and accurate interactive segmentation2010
- Exploiting simple hierarchies for unsupervised human behavior analysis2010
- Unsupervised Learning of Probabilistic Context-Free Grammar using Iterative Biclustering2008
- A Stochastic Grammar of ImagesFoundations and Trends in Computer Graphics and Vision, 2007
- From frequent itemsets to semantically meaningful visual patterns2007
- Learning Motion Categories using both Semantic and Structural Information2007
- Trajectory Series Analysis based Event Rule Induction for Visual Surveillance2007
- Conditional models for contextual human motion recognitionComputer Vision and Image Understanding, 2006
- Recognition of visual activities and interactions by stochastic parsingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000