Modeling individual and group actions in meetings with layered HMMs
Authors:
- Dong ZhangDZDong Zhang
- D. Gatica-PerezDGD. Gatica-Perez
- S. BengioSBS. Bengio
- I. McCowanIMI. McCowan
- 15 May 2006
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 8 (3) , 509-520
Abstract
We address the problem of recognizing sequences of human interaction patterns in meetings, with the goal of structuring them in semantic terms. The investigated patterns are inherently group-based (defined by the individual activities of meeting participants, and their interplay), and multimodal (as captured by cameras and microphones). By defining a proper set of individual actions, group actions can be modeled as a two-layer process, one that models basic individual activities from low-level audio-visual (AV) features,and another one that models the interactions. We propose a two-layer hidden Markov model (HMM) framework that implements such concept in a principled manner, and that has advantages over previous works. First, by decomposing the problem hierarchically, learning is performed on low-dimensional observation spaces, which results in simpler models. Second, our framework is easier to interpret, as both individual and group actions have a clear meaning, and thus easier to improve. Third, different HMMs can be used in each layer, to better reflect the nature of each subproblem. Our framework is general and extensible, and we illustrate it with a set of eight group actions, using a public 5-hour meeting corpus. Experiments and comparison with a single-layer HMM baseline system show its validity.Keywords
This publication has 30 references indexed in Scilit:
- Layered representations for learning and inferring office activity from multiple sensory channelsComputer Vision and Image Understanding, 2004
- Relationship between dialogue acts and hot spots in meetings2004
- Tracking focus of attention in meetings2003
- Detection of agreement vs. disagreement in meetings2003
- Group Discussion as Interactive Dialogue or as Serial Monologue: The Influence of Group SizePsychological Science, 2000
- A Bayesian computer vision system for modeling human interactionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Learning Behaviour Models of Human Activities1999
- Pfinder: real-time tracking of the human bodyIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997
- Speaking turns in small group interaction: A context-sensitive event sequence model.Journal of Personality and Social Psychology, 1988
- Error bounds for convolutional codes and an asymptotically optimum decoding algorithmIEEE Transactions on Information Theory, 1967