Features for multimodal emotion recognition: An extensive study
- 1 June 2010
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 23268123,p. 90-95
Abstract
The ability to recognize emotions in natural human communications is known to be very important for mankind. In recent years, a considerable number of researchers have investigated techniques allowing computer to replicate this capability by analyzing both prosodic (voice) and facial expressions. The applications of the resulting systems are manifold and range from gaming to indexing and retrieval, through chat and health care. No study has, to the best of our knowledge, ever reported results comparing the effectiveness of several features for automatic emotion recognition. In this work, we present an extensive study conducted on feature selection for automatic, audio-visual, real-time, and person independent emotion recognition. More than 300,000 different neural networks have been trained in order to compare the performances of 64 features and 11 different sets of features with 450 different analysis settings. Results show that: 1) to build an optimal emotion recognition system, different emotions should be classified via different features and 2) different features, in general, require different processing.Keywords
This publication has 13 references indexed in Scilit:
- Towards multimodal emotion recognition2010
- Evidence Theory-Based Multimodal Emotion Recognition2009
- A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous ExpressionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Facial Expression Recognition: A Fully Integrated Approach2007
- The eNTERFACE’05 Audio-Visual Emotion Database2006
- Analysis of emotion recognition using facial expressions, speech and multimodal information2004
- Emotion recognition from physiological signals using wireless sensors for presence technologiesCognition, Technology & Work, 2004
- Facial expression recognition from video sequences2003
- Automatic facial expression analysis: a surveyPattern Recognition, 2003
- A new pan-cultural facial expression of emotionMotivation and Emotion, 1986