Music Theory-Inspired Acoustic Representation for Speech Emotion Recognition
- 26 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 2534-2547
Abstract
This research presents a music theory-inspired acoustic representation (hereafter, MTAR) to address improved speech emotion recognition. The recognition of emotion in speech and music is developed in parallel, yet a relatively limited understanding of MTAR for interpreting speech emotions is involved. In the present study, we use music theory to study representative acoustics associated with emotion in speech from vocal emotion expressions and auditory emotion perception domains. In experiments assessing the role and effectiveness of the proposed representation in classifying discrete emotion categories and predicting continuous emotion dimensions, it shows promising performance compared with extensively used features for emotion recognition based on the spectrogram, Mel-spectrogram, Mel-frequency cepstral coefficients, VGGish, and the large baseline feature sets of the INTERSPEECH challenges. This proposal opens up a novel research avenue in developing a computational acoustic representation of speech emotion via music theory.Keywords
Funding Information
- Key Research and Development Program of Hainan Province (ZDYF2021GXJS017)
- National Natural Science Foundation of China (82160345, 62201571)
- Key science and technology plan project of Haikou (2021-016)
This publication has 126 references indexed in Scilit:
- Music Cognition: A Developmental PerspectiveTopics in Cognitive Science, 2012
- Children's recognition of emotions from vocal cuesBritish Journal of Developmental Psychology, 2012
- The Relationship between Music and LanguageFrontiers in Psychology, 2012
- Survey on speech emotion recognition: Features, classification schemes, and databasesPattern Recognition, 2011
- Musical intervals and relative pitch: Frequency resolution, not interval resolution, is specialThe Journal of the Acoustical Society of America, 2010
- Music perception, pitch, and the auditory systemCurrent Opinion in Neurobiology, 2008
- fMRI Evidence for a Cortical Hierarchy of Pitch Pattern ProcessingPLOS ONE, 2008
- Musical intervals in speechProceedings of the National Academy of Sciences, 2007
- Implicit Investigations of Tonal Knowledge in Nonmusician ListenersAnnals of the New York Academy of Sciences, 2005
- Timbral hierarchiesContemporary Music Review, 1987