Emotion recognition using spatial-temporal EEG features through convolutional graph attention network
Open Access
- 1 February 2023
- journal article
- research article
- Published by IOP Publishing in Journal of Neural Engineering
- Vol. 20 (1) , 016046
Abstract
Objective. Constructing an efficient human emotion recognition model based on electroencephalogram (EEG) signals is significant for realizing emotional brain–computer interaction and improving machine intelligence. Approach. In this paper, we present a spatial-temporal feature fused convolutional graph attention network (STFCGAT) model based on multi-channel EEG signals for human emotion recognition. First, we combined the single-channel differential entropy (DE) feature with the cross-channel functional connectivity (FC) feature to extract both the temporal variation and spatial topological information of EEG. After that, a novel convolutional graph attention network was used to fuse the DE and FC features and further extract higher-level graph structural information with sufficient expressive power for emotion recognition. Furthermore, we introduced a multi-headed attention mechanism in graph neural networks to improve the generalization ability of the model. Main results. We evaluated the emotion recognition performance of our proposed model on the public SEED and DEAP datasets, which achieved a classification accuracy of 99.11% ± 0.83% and 94.83% ± 3.41% in the subject-dependent and subject-independent experiments on the SEED dataset, and achieved an accuracy of 91.19% ± 1.24% and 92.03% ± 4.57% for discrimination of arousal and valence in subject-independent experiments on DEAP dataset. Notably, our model achieved state-of-the-art performance on cross-subject emotion recognition tasks for both datasets. In addition, we gained insight into the proposed frame through both the ablation experiments and the analysis of spatial patterns of FC and DE features. Significance. All these results prove the effectiveness of the STFCGAT architecture for emotion recognition and also indicate that there are significant differences in the spatial-temporal characteristics of the brain under different emotional states.Keywords
Funding Information
- Japan Society for the Promotion of Science (20K11883)
- National Natural Science Foundation of China (61876126)
This publication has 39 references indexed in Scilit:
- Reading words and other people: A comparison of exception word, familiar face and affect processing in the left and right temporal variants of primary progressive aphasiaCortex, 2016
- Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural NetworksIEEE Transactions on Autonomous Mental Development, 2015
- A tale of two hemispheres: Contrasting socioemotional dysfunction in right- versus left-lateralised semantic dementiaDementia & Neuropsychologia, 2013
- DEAP: A Database for Emotion Analysis ;Using Physiological SignalsIEEE Transactions on Affective Computing, 2011
- The role of asymmetric frontal cortical activity in emotion-related phenomena: A review and updateBiological Psychology, 2010
- Hemispheric Dominance for Emotions, Empathy and Social Behaviour: Evidence from Right and Left Handers with Frontotemporal Dementia.Neurocase, 2001
- Effects of experimentally-induced emotional states on frontal lobe cognitive task performanceNeuropsychologia, 1999
- The temporal variant of frontotemporal dementiaBrain, 1997
- Lateral brain function, emotion, and conceptualization.Psychological Bulletin, 1981
- Differing emotional response from right and left hemispheresNature, 1976