Dual-Encoder VAE-GAN With Spatiotemporal Features for Emotional EEG Data Augmentation
Open Access
- 13 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Vol. 31 (15344320) , 2018-2027
Abstract
The current data scarcity problem in EEG-based emotion recognition tasks leads to difficulty in building high-precision models using existing deep learning methods. To tackle this problem, a dual encoder variational autoencoder-generative adversarial network (DEVAE-GAN) incorporating spatiotemporal features is proposed to generate high-quality artificial samples. First, EEG data for different emotions are preprocessed as differential entropy features under five frequency bands and divided into segments with a 5s time window. Secondly, each feature segment is processed in two forms: the temporal morphology data and the spatial morphology data distributed according to the electrode position. Finally, the proposed dual encoder is trained to extract information from these two features, concatenate the two pieces of information as latent variables, and feed them into the decoder to generate artificial samples. To evaluate the effectiveness, a systematic experimental study was conducted in this work on the SEED dataset. First, the original training dataset is augmented with different numbers of generated samples; then, the augmented training datasets are used to train the deep neural network to construct the sentiment model. The results show that the augmented datasets generated by the proposed method have an average accuracy of 97.21% on all subjects, which is a 5% improvement compared to the original dataset, and the similarity between the generated data and the original data distribution is proved. These results demonstrate that our proposed model can effectively learn the distribution of raw data to generate high-quality artificial samples, which can effectively train a high-precision affective model.Keywords
Funding Information
- National Natural Science Foundation of China (61372023, 61971168, 62071161, 62271181)
- Graduate research innovation fund of Hangzhou Dianzi University (CXJJ2022156)
This publication has 32 references indexed in Scilit:
- Hierarchical Convolutional Neural Networks for EEG-Based Emotion RecognitionCognitive Computation, 2017
- Rotational data augmentation for electroencephalographic data2017
- Identifying Stable Patterns over Time for Emotion Recognition from EEGIEEE Transactions on Affective Computing, 2017
- Emotion Recognition based on EEG using LSTM Recurrent Neural NetworkInternational Journal of Advanced Computer Science and Applications, 2017
- Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer InterfacesProceedings of the IEEE, 2015
- Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural NetworksIEEE Transactions on Autonomous Mental Development, 2015
- A brain-computer interface for potential non-verbal facial communication based on EEG signals related to specific emotionsFrontiers in Neuroscience, 2014
- Using body movement and posture for emotion detection in non-acted scenarios2012
- A hierarchical positive and negative emotion understanding system based on integrated analysis of visual and brain signalsNeurocomputing, 2010
- Emotion recognition from expressions in face, voice, and body: The Multimodal Emotion Recognition Test (MERT).Emotion, 2009