A graphical model for audiovisual object tracking
Authors:
- N. JojicNJN. Jojic
- H. AttiasHAH. Attias
- 20 June 2003
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 25 (7) , 828-836
Abstract
We present a new approach to modeling and processing multimedia data. This approach is based on graphical models that combine audio and video variables. We demonstrate it by developing a new algorithm for tracking a moving object in a cluttered, noisy scene using two microphones and a camera. Our model uses unobserved variables to describe the data in terms of the process that generates them. It is therefore able to capture and exploit the statistical structure of the audio and video data separately, as well as their mutual dependencies. Model parameters are learned from data via an EM algorithm, and automatic calibration is performed as part of this procedure. Tracking is done by Bayesian inference of the object location from data. We demonstrate successful performance on multimedia clips captured in real world scenarios using off-the-shelf equipment.Keywords
This publication has 28 references indexed in Scilit:
- Transformation-invariant clustering using the EM algorithmIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- Distributed meetings2002
- Joint Audio-Visual Tracking Using Particle FiltersEURASIP Journal on Advances in Signal Processing, 2002
- Tracking multiple talkers using microphone-array measurements2002
- Active speech source localization by a dual coarse-to-fine search2002
- A multisensor-based collision avoidance system with application to a military HMMWV2002
- Audio-visual tracking for natural interactivity1999
- Blind Source Separation and Deconvolution: The Dynamic Component Analysis AlgorithmNeural Computation, 1998
- An Introduction to Variational Methods for Graphical Models1998
- A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants1998