Active and dynamic information fusion for facial expression understanding from image sequences
Authors:
- 21 March 2005
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 27 (5) , 699-714
Abstract
This paper explores the use of multisensory information fusion technique with dynamic Bayesian networks (DBN) for modeling and understanding the temporal behaviors of facial expressions in image sequences. Our facial feature detection and tracking based on active IR illumination provides reliable visual information under variable lighting and head motion. Our approach to facial expression recognition lies in the proposed dynamic and probabilistic framework based on combining DBN with Ekman's facial action coding system (FACS) for systematically modeling the dynamic and stochastic behaviors of spontaneous facial expressions. The framework not only provides a coherent and unified hierarchical probabilistic framework to represent spatial and temporal information related to facial expressions, but also allows us to actively select the most informative visual cues from the available information sources to minimize the ambiguity in recognition. The recognition of facial expressions is accomplished by fusing not only from the current visual observations, but also from the previous visual evidences. Consequently, the recognition becomes more robust and accurate through explicitly modeling temporal behavior of facial expression. In this paper, we present the theoretical foundation underlying the proposed probabilistic and dynamic framework for facial expression modeling and understanding. Experimental results demonstrate that our approach can accurately and robustly recognize spontaneous facial expressions from an image sequence under different conditions.Keywords
This publication has 40 references indexed in Scilit:
- A probabilistic framework for embedded face and facial expression recognition2003
- Facial expression understanding in image sequences using dynamic and active visual information fusion2003
- Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron2002
- LAFTER: lips and face real time tracker2002
- Modeling and Animating Realistic Faces from ImagesInternational Journal of Computer Vision, 2002
- Visual Estimation and Compression of Facial Motion Parameters—Elements of a 3D Model-Based Video Coding SystemInternational Journal of Computer Vision, 2002
- Detection, tracking, and classification of action units in facial expressionRobotics and Autonomous Systems, 2000
- Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image MotionInternational Journal of Computer Vision, 1997
- Human expression recognition from motion using a radial basis function network architectureIEEE Transactions on Neural Networks, 1996
- Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face.Journal of Personality and Social Psychology, 1979