BCAF: Bimodal connection attention fusion for robust emotion recognition from speech and text

Authors:
Publisher website
Google Scholar
Abstract
Bimodal speech emotion recognition is challenging due to the difficulty of capturing subtle emotional cues and effectively modeling complex interactions between audio and text. Existing methods often overlook the importance of modality connections and struggle to filter noisy or misleading cross-modal signals. To address these issues, we propose a novel Bimodal Connection Attention Fusion (BCAF) framework that explicitly models modality-specific representations and their interactions. BCAF includes three key modules: an interactive connection network that learns modality connections through an encoderdecoder structure; a bimodal attention network that captures intra- and inter-modal dependencies; and a correlative attention network that filters cross-modal noise to improve robustness. Extensive evaluations across benchmark datasets show that BCAF achieves superior performance over existing methods, highlighting its strength in modeling detailed modality connections and interactions for emotion recognition.
Funding Information
  • Queen Mary University of London

This publication has 51 references indexed in Scilit: