Abstract
Image–text multimodal sentiment analysis aims to integrate textual and visual information to comprehensively understand sentiment expressions in complex scenarios. However, existing methods focus on cross-modal feature interaction and fusion, and still have difficulty capturing effective sentiment cues in scenarios involving insufficient textual semantics, implicit visual affective cues, and inconsistent sentiment expressions between text and image. To address these issues, this paper proposes an image–text multimodal sentiment analysis method with large model-generated descriptive semantics and difference-aware gated fusion. Specifically, a large model generates semantic descriptions for image–text pairs, from which an enhanced semantic view is constructed to supplement implicit or insufficiently expressed sentiment cues in the original modalities. An original-enhanced dual-branch structure models the original image–text evidence and enhanced semantic evidence separately. To improve semantic consistency between the two branches, a cross-branch semantic alignment mechanism is introduced to reduce semantic shifts caused by enhanced information. In the fusion stage, difference-aware gated fusion and residual compensation are employed to adaptively balance branch contributions while preserving discriminative branch differences. Experimental results on the MVSA-Single and MVSA-Multiple datasets show that the proposed method improves performance in image–text multimodal sentiment classification, thereby validating the effectiveness of combining semantic enhancement with difference-aware modeling.
Funding Information
  • Xinjiang Uygur Autonomous Region Innovation Environment (Talent and Base) Construction Special Program—Natural Science Program (2025D03032)
  • Undergraduate Education and Teaching Research and Reform Project of Universities in the Xinjiang Uygur Autonomous Region (XJGXPTJG-2026077)
  • Xinjiang Normal University (XJNUQB2022-22)
  • Xinjiang Normal University (XJNU-ZHJY202403)
  • National Natural Science Foundation of China (61662081)
  • National Social Science Fund of China (14AZD11)

This publication has 34 references indexed in Scilit: