Prompt Fusion and Aspect-Oriented Filtration for Aspect-Based Multimodal Sentiment Analysis
Authors:
- 6 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
Currently, aspect-based multimodal sentiment analysis (ABMSA) remains a highly hot research field, aiming to leverage various modalities such as images and text to determine the sentiment orientation of viewpoint entities. Although deep learning methods have made significant progress in this field, some challenges still exist: Image information unrelated to the viewpoint entity can introduce noise, insufficient interaction during modal fusion. To solve these problems, we propose a novel ABMSA model with prompt fusion and noise filtration. Specifically, we introduced an Image Noise Filtering (INF) module that can filter the extracted image features to reduce the noise introduced by the irrelevant between viewpoint entity and the image. Additionally, we implemented Multimodal Prompt Fusion (MPF) module to enhance the interaction between modalities, addressing the issue of minimal effective features related to viewpoint entity during the fusion stage of text and image features. Experimental results show that the model outperforms the baseline model on two public datasets, Twitter-2015 and Twitter-2017, demonstrating that our proposed approach effectively enhances the accuracy of aspect-based multimodal sentiment analysis tasks.Keywords
Funding Information
- National Natural Science Foundation of China
This publication has 21 references indexed in Scilit:
- Prompt Fusion Interaction Transformer For Aspect-Based Multimodal Sentiment Analysis2024
- AMIFN: Aspect-guided multi-view interactions and fusion network for multimodal aspect-based sentiment analysisNeurocomputing, 2024
- MOCOLNet: A Momentum Contrastive Learning Network for Multimodal Aspect-Level Sentiment AnalysisIEEE Transactions on Knowledge and Data Engineering, 2023
- Multimodal Sentiment Analysis: A Survey of Methods, Trends, and ChallengesACM Computing Surveys, 2023
- A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and ChallengesIEEE Transactions on Knowledge and Data Engineering, 2022
- Hierarchical Interactive Multimodal Transformer for Aspect-Based Multimodal Sentiment AnalysisIEEE Transactions on Affective Computing, 2022
- Entity-Sensitive Attention and Fusion Network for Entity-Level Multimodal Sentiment ClassificationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019
- VistaNet: Visual Aspect Attention Network for Multimodal Sentiment AnalysisProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment AnalysisProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Gated Neural Networks for Targeted Sentiment AnalysisProceedings of the AAAI Conference on Artificial Intelligence, 2016