Reward-Guided Dynamic Fusion and Modality Decoupling for Enhanced Multimodal Sentiment Analysis
Open Access
- 24 April 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (9) , 1813
Abstract
Multimodal Sentiment Analysis (MSA) integrates multiple modalities to better understand human emotions. However, existing methods often neglect heterogeneity among modal features, causing redundancy and inconsistencies. Additionally, the dynamic interplay between modalities is frequently ignored during fusion, limiting performance. To address these issues, we propose Reward-Guided Dynamic Fusion and Modality Decoupling (RDFD). RDFD includes two key components: (1) a feature decoupling module that separates modality-specific and modality-shared features, reducing redundancy and conflicts; (2) a Reward-Guided Dynamic Fusion module that adaptively selects guiding modalities to enhance modality-specific representations and enable flexible fusion. Experiments on the CMU-MOSI and CMU-MOSEI datasets show that RDFD achieves state-of-the-art performance, demonstrating its effectiveness in advancing Multimodal Sentiment Analysis.Keywords
Funding Information
- National Key Research and Development Program of China (2021ZD0110400)
- National Key Research and Development Program of China (2021ZD0110404)
- National Natural Science Foundation of China (62271377)
- National Natural Science Foundation of China (62201407)
- Key Research and Development Program of Shaanxi Province (2021ZDLGY01-06)
- Key Research and Development Program of Shaanxi Province (2022ZDLGY01-12)
- Key Research and Development Program of Shaanxi Province (2023YBGY244)
- Key Research and Development Program of Shaanxi Province (2023QCYLL28)
- Key Research and Development Program of Shaanxi Province (2024GX-ZDCYL-02-08)
- Key Research and Development Program of Shaanxi Province (2024GX-ZDCYL-02-17)
- Xi’an Social Sciences Planning Fund Project (25FZ63)
- Humanities and Social Sciences Research Project of the Shaanxi Provincial Department of Education (20JK0377)
This publication has 27 references indexed in Scilit:
- Hallusionbench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models2024
- FDR-MSA: Enhancing multimodal sentiment analysis through feature disentanglement and reconstructionKnowledge-Based Systems, 2024
- Text-Oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequences2024
- Progress, achievements, and challenges in multimodal sentiment analysis using deep learning: A surveyApplied Soft Computing, 2023
- Disentanglement Translation Network for multimodal sentiment analysisInformation Fusion, 2023
- TETFN: A text enhanced transformer fusion network for multimodal sentiment analysisPattern Recognition, 2022
- Attention is not Enough: Mitigating the Distribution Discrepancy in Asynchronous Multimodal Sequence Fusion2021
- Progressive Modality Reinforcement for Human Multimodal Emotion Recognition from Unaligned Multimodal Sequences2021
- Affective video content analysis based on multimodal data fusion in heterogeneous networksInformation Fusion, 2019
- Sentiment analysis: A literature review2012