Abstract
Multimodal Sentiment Analysis (MSA) integrates multiple modalities to better understand human emotions. However, existing methods often neglect heterogeneity among modal features, causing redundancy and inconsistencies. Additionally, the dynamic interplay between modalities is frequently ignored during fusion, limiting performance. To address these issues, we propose Reward-Guided Dynamic Fusion and Modality Decoupling (RDFD). RDFD includes two key components: (1) a feature decoupling module that separates modality-specific and modality-shared features, reducing redundancy and conflicts; (2) a Reward-Guided Dynamic Fusion module that adaptively selects guiding modalities to enhance modality-specific representations and enable flexible fusion. Experiments on the CMU-MOSI and CMU-MOSEI datasets show that RDFD achieves state-of-the-art performance, demonstrating its effectiveness in advancing Multimodal Sentiment Analysis.
Funding Information
  • National Key Research and Development Program of China (2021ZD0110400)
  • National Key Research and Development Program of China (2021ZD0110404)
  • National Natural Science Foundation of China (62271377)
  • National Natural Science Foundation of China (62201407)
  • Key Research and Development Program of Shaanxi Province (2021ZDLGY01-06)
  • Key Research and Development Program of Shaanxi Province (2022ZDLGY01-12)
  • Key Research and Development Program of Shaanxi Province (2023YBGY244)
  • Key Research and Development Program of Shaanxi Province (2023QCYLL28)
  • Key Research and Development Program of Shaanxi Province (2024GX-ZDCYL-02-08)
  • Key Research and Development Program of Shaanxi Province (2024GX-ZDCYL-02-17)
  • Xi’an Social Sciences Planning Fund Project (25FZ63)
  • Humanities and Social Sciences Research Project of the Shaanxi Provincial Department of Education (20JK0377)