Facial Emotion Recognition Using Hybrid Features
Open Access
- 18 February 2020
- journal article
- research article
- Published by MDPI AG in Informatics
- Vol. 7 (1) , 6
Abstract
Facial emotion recognition is a crucial task for human-computer interaction, autonomous vehicles, and a multitude of multimedia applications. In this paper, we propose a modular framework for human facial emotions’ recognition. The framework consists of two machine learning algorithms (for detection and classification) that could be trained offline for real-time applications. Initially, we detect faces in the images by exploring the AdaBoost cascade classifiers. We then extract neighborhood difference features (NDF), which represent the features of a face based on localized appearance information. The NDF models different patterns based on the relationships between neighboring regions themselves instead of considering only intensity information. The study is focused on the seven most important facial expressions that are extensively used in day-to-day life. However, due to the modular design of the framework, it can be extended to classify N number of facial expressions. For facial expression classification, we train a random forest classifier with a latent emotional state that takes care of the mis-/false detection. Additionally, the proposed method is independent of gender and facial skin color for emotion recognition. Moreover, due to the intrinsic design of NDF, the proposed method is illumination and orientation invariant. We evaluate our method on different benchmark datasets and compare it with five reference methods. In terms of accuracy, the proposed method gives 13% and 24% better results than the reference methods on the static facial expressions in the wild (SFEW) and real-world affective faces (RAF) datasets, respectively.Keywords
This publication has 55 references indexed in Scilit:
- Robust Representation and Recognition of Facial Emotions Using Extreme Sparse LearningIEEE Transactions on Image Processing, 2015
- Emotional States Associated with MusicACM Transactions on Interactive Intelligent Systems, 2015
- Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift AdaptationThe Scientific World Journal, 2014
- Smartphone-Based Recognition of States and State Changes in Bipolar Disorder PatientsIEEE Journal of Biomedical and Health Informatics, 2014
- Description of interest regions with local binary patternsPattern Recognition, 2009
- Multi-class AdaBoostStatistics and Its Interface, 2009
- Adaptive Thresholding using the Integral ImageJournal of Graphics Tools, 2007
- Isolated word recognition with the Liquid State Machine: a case studyInformation Processing Letters, 2005
- A multiple-cascade-classifier system for a robust and partially unsupervised updating of land-cover mapsIEEE Transactions on Geoscience and Remote Sensing, 2002