GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Speech Emotion Recognition
Preprint
Abstract
Contrastive learning based pretraining methods have recently exhibited impressive success in diverse fields. In this paper, we propose GEmo-CLAP, a kind of efficient gender-attribute-enhanced contrastive language-audio pretraining (CLAP) model for speech emotion recognition. To be specific, we first build an effective emotion CLAP model Emo-CLAP for emotion recognition, utilizing various self-supervised learning based pre-trained models. Then, considering the importance of the gender attribute in speech emotion modeling, two GEmo-CLAP approaches are further proposed to integrate the emotion and gender information of speech signals, forming more reasonable objectives. Extensive experiments on the IEMOCAP corpus demonstrate that our proposed two GEmo-CLAP approaches consistently outperform the baseline Emo-CLAP with different pre-trained models, while also achieving superior recognition performance compared with other state-of-the-art methods.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition2023
- Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence GapIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognition2022
- Multi-Corpus Speech Emotion Recognition for Unseen Corpus Using Corpus-Wise Weights in Classification Loss2022
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden UnitsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Emotion Recognition from Speech Using wav2vec 2.0 Embeddings2021
- Speech Technology for Healthcare: Opportunities, Challenges, and State of the ArtIEEE Reviews in Biomedical Engineering, 2020
- Customer Satisfaction Estimation in Contact Center Calls Based on a Hierarchical Multi-Task ModelIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020