Multi-Channel Pose-Aware Convolution Neural Networks for Multi-View Facial Expression Recognition
Authors:
- Yuanyuan LiuYLYuanyuan Liu
- Jiabei ZengJZJiabei Zeng
- Shiguang ShanSSShiguang Shan
- Zhuo ZhengZZZhuo Zheng
- 1 May 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 458-465
Abstract
Although tremendous strides have been made in facial expression recognition(FER), recognizing facial expressions in non-frontal views remains an open challenge due to the limited access to large scale training data with various poses. To make full use of the limited data, we propose a novel multi-channel pose-aware convolution neural network (MPCNN) that consists of three parts: the multi-channel feature extraction, jointly multi-scale feature fusion, and the pose-aware recognition. The feature extraction part has 3 sub-CNNs and it learns convolutional features from different features. The joint fusion part fuses multi-scale features to enhance high-level feature representation in a hierarchical way. The fused features are fed to the pose-aware recognition part that includes pose-specific recognition branches and a pose estimation sub-network. According to the estimated pose, MPCNN finally classifies the facial expression through a conditional weighted combination of the pose-specific recognition branches. MPCNN is end-to-end trainable by minimizing the joint loss of pose and expression recognition. We evaluated the proposed method on two public multi-view FER datasets (BU-3DFE and KDEF) and a FER dataset in the wild (SFEW). The experimental results demonstrate that MPCNN outperforms the state-of-the-art FER methods with both within-dataset and cross-dataset settings.Keywords
This publication has 17 references indexed in Scilit:
- Pose-specific non-linear mappings in feature space towards multiview facial expression recognitionImage and Vision Computing, 2017
- Going deeper in facial expression recognition using deep neural networks2016
- Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition2015
- Fast R-CNN2015
- Multi-pose facial expression recognition based on SURF boosting2015
- Deep Face Recognition2015
- Multi-View Facial Expression Recognition Based on Group Sparse Reduced-Rank RegressionIEEE Transactions on Affective Computing, 2014
- Local binary patterns for multi-view facial expression recognitionComputer Vision and Image Understanding, 2011
- Coupled Gaussian Process Regression for Pose-Invariant Facial Expression Recognition2010
- Facial expression recognition based on Local Binary Patterns: A comprehensive studyImage and Vision Computing, 2009