Rethinking Cross-Corpus Speech Emotion Recognition Benchmarking: Are Paralinguistic Pre-Trained Representations Sufficient?
Authors:
- Orchid Chetia PhukanOPOrchid Chetia Phukan
- GirishGIGirish
Preprint
Abstract
Recent benchmarks evaluating pre-trained models (PTMs) for cross-corpus speech emotion recognition (SER) have overlooked PTM pre-trained for paralinguistic speech processing (PSP), raising concerns about their reliability, since SER is inherently a paralinguistic task. We hypothesize that PSP-focused PTM will perform better in cross-corpus SER settings. To test this, we analyze state-of-the-art PTMs representations including paralinguistic, monolingual, multilingual, and speaker recognition. Our results confirm that TRILLsson (a paralinguistic PTM) outperforms others, reinforcing the need to consider PSP-focused PTMs in cross-corpus SER benchmarks. This study enhances benchmark trustworthiness and guides PTMs evaluations for reliable cross-corpus SER.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Evaluating Self-Supervised Speech Representations for Speech Emotion RecognitionIEEE Access, 2022
- XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale2022
- TRILLsson: Distilled Universal Paralinguistic Speech Representations2022
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- Emotion Recognition from Speech Using wav2vec 2.0 Embeddings2021
- SUPERB: Speech Processing Universal PERformance Benchmark2021
- Speech Emotion Recognition for Performance InteractionJournal of the Audio Engineering Society, 2018
- The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American EnglishPLOS ONE, 2018
- X-Vectors: Robust DNN Embeddings for Speaker Recognition2018
- CREMA-D: Crowd-Sourced Emotional Multimodal Actors DatasetIEEE Transactions on Affective Computing, 2014