Recognition of Affect in the Wild Using Deep Neural Networks
Authors:
- Dimitrios KolliasDimitrios Kollias
- Mihalis A. NicolaouMNMihalis A. Nicolaou
- Irene KotsiaIKIrene Kotsia
- Guoying ZhaoGZGuoying Zhao
- Stefanos ZafeiriouSZStefanos Zafeiriou
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1972-1979
Abstract
In this paper we utilize the first large-scale "in-the-wild" (Aff-Wild) database, which is annotated in terms of the valence-arousal dimensions, to train and test an end-to-end deep neural architecture for the estimation of continuous emotion dimensions based on visual cues. The proposed architecture is based on jointly training convolutional (CNN) and recurrent neural network (RNN) layers, thus exploiting both the invariant properties of convolutional features, while also modelling temporal dynamics that arise in human behaviour via the recurrent layers. Various pre-trained networks are used as starting structures which are subsequently appropriately fine-tuned to the Aff-Wild database. Obtained results show premise for the utilization of deep architectures for the visual analysis of human behaviour in terms of continuous emotion dimensions and analysis of different types of affect.Keywords
This publication has 16 references indexed in Scilit:
- How deep neural networks can improve emotion recognition on video data2016
- Deep Residual Learning for Image Recognition2016
- Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network2016
- Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition2015
- AVEC 20152015
- Deep learningNature, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Automatic analysis of facial expressions: the state of the artIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Long Short-Term MemoryNeural Computation, 1997