Speech Emotion Recognition in Conversations Using Artificial Intelligence: A Systematic Review and Meta-Analysis
Authors:
Preprint
- 27 September 2023
- preprint
- Published by Springer Nature in Research Square
Abstract
Purpose: Manifestations of emotion in social conversational interactions stand at a focal point in the rapidly growing affective computing area, with applications in healthcare, education and human-computer interaction. Artificial intelligence (AI) holds great potential in modelling the challenging dynamic nature of affect in speech conversation. In this paper, we analyze and criticize latest trends and open problems through a systematic review and multi-subgroup meta-analysis of AI approaches for emotion recognition in conversation (ERC). Methods: We adopt the PRISMA-DTA guidelines toward analysis of AI-driven speech ERC. A comprehensive database search through predefined query strings and selection criteria allowed for data extraction of essential diagnostic performance parameters. We analyze salient patterns related to methodological quality and risk of bias. Univariate random-effects models are then designed with a multi-subgroup perspective, centered around affective annotations models, while encompassing the ERC parameters of modalities, feature extraction and conversation style. Results: 51 studies were systematically reviewed for qualitative analysis, whereas 27 articles were included in the meta-analysis. Diagnostic test performance manifested with high heterogeneity, with intriguing insights regarding affective state annotation, input modality, feature extraction methods, and dataset conversation style. Conclusion: Our research contributes fine-grained insights as recommendations that tackle open-problems in ERC. While providing valuable information on diagnostic performance of AI in speech ERC, we underscore the imperative need for further advancements in annotations and models capable of handling diverse emotional expressions.Keywords
All Related Versions
- Version 1, 2023-09-27, Research Square
- Published version: Artificial Intelligence Review, 58 (7), 1.
This publication has 29 references indexed in Scilit:
- K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention LabelsScientific Data, 2023
- A Multitask learning model for multimodal sarcasm, sentiment and emotion recognition in conversationsInformation Fusion, 2023
- DECN: Dialogical emotion correction network for conversational emotion recognitionNeurocomputing, 2021
- A modified feature selection method based on metaheuristic algorithms for speech emotion recognitionApplied Acoustics, 2021
- Heart sound signals can be used for emotion recognitionScientific Reports, 2019
- Introduction, comparison, and validation of Meta‐Essentials: A free and simple tool for meta‐analysisResearch Synthesis Methods, 2017
- Modelling speech emotion recognition using logistic regression and decision treesInternational Journal of Speech Technology, 2017
- Knowledge-based Framework for Intelligent Emotion Recognition in Spontaneous SpeechProcedia Computer Science, 2016
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- Categorical perception of facial expressionsCognition, 1992