Multimodal fusion of audio, scene, and face features for first impression estimation
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 43-48
Abstract
Affective computing, particularly emotion and personality trait recognition, is of increasing interest in many research disciplines. The interplay of emotion and personality shows itself in the first impression left on other people. Moreover, the ambient information, e.g. the environment and objects surrounding the subject, also affect these impressions. In this work, we employ pre-trained Deep Convolutional Neural Networks to extract facial emotion and ambient information from images for predicting apparent personality. We also investigate Local Gabor Binary Patterns from Three Orthogonal Planes video descriptor and acoustic features extracted via the popularly used openSMILE tool. We subsequently propose classifying features using a Kernel Extreme Learning Machine and fusing their predictions. The proposed system is applied to the ChaLearn Challenge on First Impression Recognition, achieving the winning test set accuracy of 0.913, averaged over the “Big Five” personality traits.Keywords
This publication has 17 references indexed in Scilit:
- Kernel ELM and CNN Based Facial Age Estimation2016
- Contrasting and Combining Least Squares Based Learners for Emotion Recognition in the Wild2015
- Continuous Mapping of Personality Traits2014
- Automatic Recognition of Personality Traits2014
- Look! Who's Talking?2014
- The Impact of Affective Verbal Content on Predicting Personality Impressions in YouTube Videos2014
- A Multivariate Regression Approach to Personality Impression Recognition of Vloggers2014
- Feature Analysis for Computational Personality Recognition Using YouTube Personality Data set2014
- Supervised Descent Method and Its Applications to Face Alignment2013
- Extreme Learning Machine for Regression and Multiclass ClassificationIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011