Multivariate, Multi-Frequency and Multimodal: Rethinking Graph Neural Networks for Emotion Recognition in Conversation
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10761-10770
Abstract
Complex relationships of high arity across modality and context dimensions is a critical challenge in the Emotion Recognition in Conversation (ERC) task. Yet, previous works tend to encode multimodal and contextual relationships in a loosely-coupled manner, which may harm relationship modelling. Recently, Graph Neural Networks (GNN) which show advantages in capturing data relations, offer a new solution for ERC. However, existing GNN-based ERC models fail to address some general limits of GNNs, including assuming pairwise formulation and erasing high-frequency signals, which may be trivial for many applications but crucial for the ERC task. In this paper, we propose a GNN-based model that explores multivariate relationships and captures the varying importance of emotion discrepancy and commonality by valuing multi-frequency signals. We empower GNNs to better capture the inherent relationships among utterances and deliver more sufficient multimodal and contextual modelling. Experimental results show that our proposed method outperforms previous state-of-the-art works on two popular multimodal ERC datasets.Keywords
Funding Information
- National Natural Science Foundation of China (61832001,62276047)
This publication has 28 references indexed in Scilit:
- Hypergraph Neural NetworksProceedings of the AAAI Conference on Artificial Intelligence, 2019
- DialogueRNN: An Attentive RNN for Emotion Detection in ConversationsProceedings of the AAAI Conference on Artificial Intelligence, 2019
- MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations2019
- ICON: Interactive Conversational Memory Network for Multimodal Emotion Detection2018
- Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos2018
- Densely Connected Convolutional Networks2017
- The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domainsIEEE Signal Processing Magazine, 2013
- Opensmile2010
- ]Video object segmentation by hypergraph cut2009
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008