Assess and Guide: Multi-modal Fake News Detection via Decision Uncertainty
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 37-44
Abstract
The prevalence of fake news, in both visual and textual forms, has become a critical and urgent issue. Existing multi-modal fake news detection methods aim to improve detection performance by leveraging information from different modalities and integrating it into a unified, comprehensive representation. However, challenges arise when different modalities hold conflicting beliefs. This suggests the existence of information asymmetry and imbalance between various modalities, which has not been adequately recognized in previous work. In this regard, focusing on the uncertainty of each modality facilitates a more nuanced understanding of its reliability and informativeness. In this work, we distinguish among uni-modal information sources through accurate estimation of their decision uncertainty, employing this uncertainty as a confidence measure and indicator of information imbalance to guide feature fusion. In particular, 1) To provide accurate uncertainty estimation, an uncertainty learning method based on the Dirichlet distribution is employed; 2) To fuse multi-modal information, uncertainty is used as a guiding factor to govern the contribution of each modality. Extensive experiments on two real-world datasets Weibo and Twitter validate the effectiveness of the proposed method.Keywords
This publication has 23 references indexed in Scilit:
- Dual-Recommendation Disentanglement Network for View Fuzz in Action RecognitionIEEE Transactions on Image Processing, 2023
- FNR: a similarity and transformer-based approach to detect multi-modal fake news in social mediaSocial Network Analysis and Mining, 2023
- Trustworthy Long-Tailed Classification2022
- Cross-Modal Knowledge Distillation in Multi-Modal Fake News Detection2022
- Trusted Multi-View Classification With Dynamic Evidential FusionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- Deep learning for fake news detection: A comprehensive surveyAI Open, 2022
- Detecting fake news by exploring the consistency of multimodal dataInformation Processing & Management, 2021
- $\mathsf {SAFE}$: Similarity-Aware Multi-modal Fake News Detection2020
- Exploiting Multi-domain Visual Information for Fake News Detection2019
- Novel Visual and Statistical Image Features for Microblogs News VerificationIEEE Transactions on Multimedia, 2016