Speech Emotion Recognition with Dual-Sequence LSTM Architecture
Authors:
Top Cited Papers
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6474-6478
Abstract
Speech Emotion Recognition (SER) has emerged as a critical component of the next generation of human-machine interfacing technologies. In this work, we propose a new dual-level model that predicts emotions based on both MFCC features and mel-spectrograms produced from raw audio signals. Each utterance is preprocessed into MFCC features and two mel-spectrograms at different time-frequency resolutions. A standard LSTM processes the MFCC features, while a novel LSTM architecture, denoted as Dual-Sequence LSTM (DS-LSTM), processes the two mel-spectrograms simultaneously. The outputs are later averaged to produce a final classification of the utterance. Our proposed model achieves, on average, a weighted accuracy of 72.7% and an unweighted accuracy of 73.3%—a 6% improvement over current state-of-the-art unimodal models—and is comparable with multimodal models that leverage textual information as well as audio signals.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Speech emotion recognition using deep 1D & 2D CNN LSTM networksBiomedical Signal Processing and Control, 2019
- CNN+LSTM Architecture for Speech Emotion Recognition with Data Augmentation2018
- Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms2017
- Reducing the Computational Complexity of Two-Dimensional LSTMs2017
- Automatic speech emotion recognition using recurrent neural networks with local attention2017
- Exploring multidimensional lstms for large vocabulary ASR2016
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- Multi-dimensional Recurrent Neural Networks2007
- Long Short-Term MemoryNeural Computation, 1997
- Uncertainty Principles and Signal RecoverySIAM Journal on Applied Mathematics, 1989