HetEmotionNet
Authors:
Open Access
- 17 October 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1047-1056
Abstract
The research on human emotion under multimedia stimulation based on physiological signals is an emerging field and important progress has been achieved for emotion recognition based on multi-modal signals. However, it is challenging to make full use of the complementarity among spatial-spectral-temporal domain features for emotion recognition, as well as model the heterogeneity and correlation among multi-modal signals. In this paper, we propose a novel two-stream heterogeneous graph recurrent neural network, named HetEmotionNet, fusing multi-modal physiological signals for emotion recognition. Specifically, HetEmotionNet consists of the spatial-temporal stream and the spatial-spectral stream, which can fuse spatial-spectral-temporal domain features in a unified framework. Each stream is composed of the graph transformer network for modeling the heterogeneity, the graph convolutional network for modeling the correlation, and the gated recurrent unit for capturing the temporal domain or spectral domain dependency. Extensive experiments on two real-world datasets demonstrate that our proposed model achieves better performance than state-of-the-art baselines.Keywords
Funding Information
- National Natural Science Foundation of China (61603029)
- the Fundamental Research Funds for the Central Universities (2020YJS025)
- the China Scholarship Council (202007090056)
This publication has 39 references indexed in Scilit:
- Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural NetworksIEEE Transactions on Autonomous Mental Development, 2015
- EEG-based emotion classification using deep belief networks2014
- Emotional state classification from EEG data using machine learning approachNeurocomputing, 2014
- The dynamics of EEG gamma responses to unpleasant visual stimuli: From local activity to functional connectivityNeuroImage, 2012
- DEAP: A Database for Emotion Analysis ;Using Physiological SignalsIEEE Transactions on Affective Computing, 2011
- Autonomic nervous system activity in emotion: A reviewBiological Psychology, 2010
- Comparing Two Emotion Models for Deriving Affective States from Physiological Data2008
- Feature extraction from EEGs associated with emotionsArtificial Life and Robotics, 1997
- A training algorithm for optimal margin classifiers1992
- Generalized Cross-Validation as a Method for Choosing a Good Ridge ParameterTechnometrics, 1979