Learning Visual Emotion Representations From Web Data
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13103-13112
Abstract
We present a scalable approach for learning powerful visual features for emotion recognition. A critical bottleneck in emotion recognition is the lack of large scale datasets that can be used for learning visual emotion features. To this end, we curate a webly derived large scale dataset, StockEmotion, which has more than a million images. StockEmotion uses 690 emotion related tags as labels giving us a fine-grained and diverse set of emotion labels, circumventing the difficulty in manually obtaining emotion annotations. We use this dataset to train a feature extraction network, EmotionNet, which we further regularize using joint text and visual embedding and text distillation. Our experimental results establish that EmotionNet trained on the StockEmotion dataset outperforms SOTA models on four different visual emotion tasks. An aded benefit of our joint embedding training approach is that EmotionNet achieves competitive zero-shot recognition performance against fully supervised baselines on a challenging visual emotion dataset, EMOTIC, which further highlights the generalizability of the learned emotion features.Keywords
This publication has 30 references indexed in Scilit:
- The role of language in emotion: predictions from psychological constructionismFrontiers in Psychology, 2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- Predicting Ad Liking and Purchase Intent: Large-Scale Analysis of Facial Responses to AdsIEEE Transactions on Affective Computing, 2014
- On Aesthetics and Emotions in Scene Images:2014
- Convolutional Neural Networks for Sentence Classification2014
- Classification in the Presence of Label Noise: A SurveyIEEE Transactions on Neural Networks and Learning Systems, 2013
- Machine Recognition of Music EmotionACM Transactions on Intelligent Systems and Technology, 2012
- A 12-point circumplex structure of core affect.Emotion, 2011
- Effectiveness of emotion recognition training for young children with developmental delays.Journal of Early and Intensive Behavior Intervention, 2008
- Affective space is bipolar.Journal of Personality and Social Psychology, 1979