REmoNet: Reducing Emotional Label Noise via Multi-regularized Self-supervision
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2204-2213
Abstract
Emotion recognition based on electroencephalogram (EEG) has garnered increasing attention in recent years due to non-invasiveness and high reliability of EEG measurements. Despite the promising performance achieved by numerous existing methods, several challenges persist. Firstly, there is the challenge of emotional label noise, stemming from the assumption that emotions remain consistently evoked and stable throughout the entirety of video observation. Such an assumption proves difficult to uphold in practical experimental settings, leading to discrepancies between EEG signals and anticipated emotional states. In addition, there's a need for comprehensive capture of temporal-spatial-spectral characteristics of EEG signals and cope with low signal-to-noise ratio (SNR) issues. To tackle these challenges, we propose a comprehensive pipeline named REmoNet, which leverages novel self-supervised techniques and multi-regularized co-learning. Two self-supervised methods, including masked channel modeling via temporal-spectral transformation and emotion contrastive learning, are introduced to facilitate the comprehensive understanding and extraction of emotion-relevant EEG representations during pre-training. Additionally, fine-tuning with multi-regularized co-learning exploits feature-dependent information through intrinsic similarity, resulting in mitigating emotional label noise. Experimental evaluations on two public datasets demonstrate that our proposed approach, REmoNet, surpasses existing state-of-the-art methods, showcasing its effectiveness in simultaneously addressing raw EEG signals and noisy emotional labels.Keywords
This publication has 36 references indexed in Scilit:
- Context Autoencoder for Self-supervised Representation LearningInternational Journal of Computer Vision, 2023
- Masked Autoencoders Are Scalable Vision Learners2022
- Masked Feature Prediction for Self-Supervised Visual Pre-Training2022
- Multi-channel EEG-based emotion recognition in the presence of noisy labelsScience China Information Sciences, 2022
- Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization2020
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- EEGNet: a compact convolutional neural network for EEG-based brain–computer interfacesJournal of Neural Engineering, 2018
- Hierarchical Convolutional Neural Networks for EEG-Based Emotion RecognitionCognitive Computation, 2017
- Deep learning with convolutional neural networks for EEG decoding and visualizationHuman Brain Mapping, 2017
- Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network2016