Speech Emotion Recognition via Multi-Level Attention Network
- 3 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 29 (10709908) , 2278-2282
Abstract
Aiming to improve the performance of human speech emotion recognition (SER), the existing work has made great progress based on the popular mel-scale frequency cepstral coefficient (MFCC). However, the existing work rarely pays attention to the low-level emotion related features in MFCC, such as the underlying interactive relations. In this letter, we propose a novel multi-level attention network (MLAnet), which contains a multi-scale low-level feature (MLF) extractor and a multi-unit attention (MUA) module. Within the MLF extractor, we minimize the task-irrelevant information which harms the performance of SER by applying the attention mechanism. Since the features extracted by the MLF extractor contain rich domain-specific emotion information, we further present a MUA module to simultaneously weight the features in terms of time, frequency and channel dimensions. In this way, the discriminative emotion features in different dimensions can be extracted by corresponding weighting blocks. Experimental results on two benchmark datasets demonstrate that the proposed method outperforms other state-of-the-art approaches.Keywords
Funding Information
- National Key Research and Development Program of China (2020YFC1523302)
- National Natural Science Foundation of China (62002290)
This publication has 28 references indexed in Scilit:
- An Attention Pooling Based Representation Learning Method for Speech Emotion Recognition2018
- Squeeze-and-Excitation Networks2018
- The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American EnglishPLOS ONE, 2018
- Speaker Recognition for Hindi Speech Signal using MFCC-GMM ApproachProcedia Computer Science, 2018
- Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid MatchingIEEE Transactions on Multimedia, 2017
- Automatic speech emotion recognition using recurrent neural networks with local attention2017
- Survey on speech emotion recognition: Features, classification schemes, and databasesPattern Recognition, 2011
- Deconvolutive Short-Time Fourier Transform SpectrogramIEEE Signal Processing Letters, 2009
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- Combining evidence from residual phase and MFCC features for speaker recognitionIEEE Signal Processing Letters, 2005