The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements
- 14 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 14 (2) , 1334-1350
Abstract
Truly real-life data presents a strong, but exciting challenge for sentiment and emotion research. The high variety of possible ‘in-the-wild’ properties makes large datasets such as these indispensable with respect to building robust machine learning models. A sufficient quantity of data covering a deep variety in the challenges of each modality to force the exploratory analysis of the interplay of all modalities has not yet been made available in this context. In this contribution, we present MuSe-CaR, a first of its kind multimodal dataset. The data is publicly available as it recently served as the testing bed for the 1st Multimodal Sentiment Analysis Challenge, and focused on the tasks of emotion, emotion-target engagement, and trustworthiness recognition by means of comprehensively integrating the audio-visual and language modalities. Furthermore, we give a thorough overview of the dataset in terms of collection and annotation, including annotation tiers not used in this year's MuSe 2020. In addition, for one of the sub-challenges -- predicting the level of trustworthiness -- no participant outperformed the baseline model, and so we propose a simple, but highly efficient Multi-Head-Attention network that exceeds using multimodal fusion the baseline by around 0.2 CCC (almost 50 % improvement).Keywords
All Related Versions
This publication has 76 references indexed in Scilit:
- Mapping discrete and dimensional emotions onto the brain: controversies and consensusTrends in Cognitive Sciences, 2012
- Context in Emotion PerceptionCurrent Directions in Psychological Science, 2011
- Quantifying the trustworthiness of social media contentDistributed and Parallel Databases, 2011
- Sentiment strength detection in short informal textJournal of the American Society for Information Science and Technology, 2010
- Affective MedicineMethods of Information in Medicine, 2010
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance.Journal of Applied Psychology, 2007
- Challenges in real-life emotion annotation and machine learning based detectionNeural Networks, 2005
- A circumplex model of affect.Journal of Personality and Social Psychology, 1980
- Trust and Social ObjectivesEthics, 1961