Affective Computing for Large-scale Heterogeneous Multimedia Data
Authors:
Open Access
- 30 November 2019
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 15 (3s) , 1-32
Abstract
The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., images, music, and videos), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61701273, 91748129)
- U.S. Army
- Berkeley DeepDrive
- National Key R&D Program of China (2017YFC011300)
This publication has 137 references indexed in Scilit:
- Norms of valence, arousal, and dominance for 13,915 English lemmasBehavior Research Methods, 2013
- Flexible Presentation of Videos Based on Affective Content AnalysisLecture Notes in Computer Science, 2013
- Machine Recognition of Music EmotionACM Transactions on Intelligent Systems and Technology, 2012
- The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significanceBehavior Research Methods, 2011
- Survey on speech emotion recognition: Features, classification schemes, and databasesPattern Recognition, 2011
- Determination of emotional content of video clips by low-level audiovisual featuresMultimedia Tools and Applications, 2011
- A comparison of the discrete and dimensional models of emotion in musicPsychology of Music, 2010
- A systems approach to appraisal mechanisms in emotionNeural Networks, 2005
- MARSYAS: a framework for audio analysisOrganised Sound, 2000
- An argument for basic emotionsCognition and Emotion, 1992