A Novel Multimodal Speech Emotion Recognition System

Authors:

Abstract
The recognition of speech emotions is a popular study area. Speech signal is a combination of audio and text. The audio of speech is used as an input for detecting emotions in an Audio Emotion Recognition systems. The textual content of speech is used as an input for detecting emotions in Text Emotion Recognition systems. First, we focused on a unimodal technique, in which emotion recognition can be done using either audio or text. The study then shifted its focus to a multimodal approach, in which audio and text models are integrated to form a single modality. The AlexNet audio model had a 55% accuracy rate, and the text emotion Recognition model using the BERT model had 69% accuracy. The accuracy of the emotion detection system increased to 98% with the multimodal approach. The multimodal system comprises of both audio and text models, which improves the emotion recognition system.