Multimodal Fake News Detection via Evidence Retrieval and Visual Forensics with Large Vision-Language Models
Open Access
- 25 March 2026
- journal article
- research article
- Published by MDPI AG in Information
- Vol. 17 (4) , 317
Abstract
Fake news has caused significant harm and disruption across various sectors of society. With the rapid advancement of the Internet and social media platforms, both academic and industrial communities have shown growing interest in multimodal fake news detection. In this work, we propose MERF (Multimodal Evidence Retrieval and Forensics with LVLM), a unified framework for multimodal fake news detection that leverages the reasoning capabilities of Large Vision-Language Models (LVLMs). While LVLMs outperform traditional Large Language Models (LLMs) in processing multimodal content, our study reveals that their reasoning abilities remain limited in the absence of sufficient supporting evidence. MERF addresses this challenge by integrating web-based content retrieval, reverse image search, and image manipulation detection into a coherent pipeline, enabling the model to generate informed and explainable veracity judgments. Specifically, our approach performs cross-modal consistency checking, retrieves corroborative information for both textual and visual content, and applies forensic analysis to detect potential visual forgeries. The aggregated evidence is then fed into the LVLM, facilitating comprehensive reasoning and evidence-based decision-making. Experimental results on two public benchmark datasets—Weibo and Twitter—demonstrate that MERF consistently outperforms state-of-the-art baselines across all major evaluation metrics, achieving substantial improvements in accuracy, robustness, and interpretability.Keywords
Funding Information
- the national key Research and Development Program of China (2023YFC3304504)
- the 2024 Xizang Autonomous Region Central Guided Local Science and Technology Development Fund Project (XZ202401YD0015)
- the Basic Research Program for Natural Science of Guizhou Province (QIANKEHEJICHU-ZD[2025]-043)
This publication has 31 references indexed in Scilit:
- IMFND: In-context multimodal fake news detection with large visual-language modelsKnowledge-Based Systems, 2025
- Combating misinformation in the age of LLMs: Opportunities and challengesAI Magazine, 2024
- QMFND: A quantum multimodal fusion-based fake news detection model for social mediaInformation Fusion, 2023
- MFIR: Multimodal fusion and inconsistency reasoning for explainable fake news detectionInformation Fusion, 2023
- FNR: a similarity and transformer-based approach to detect multi-modal fake news in social mediaSocial Network Analysis and Mining, 2023
- Content-Based Fake News Detection With Machine and Deep Learning: a Systematic ReviewNeurocomputing, 2023
- Fake news, disinformation and misinformation in social media: a reviewSocial Network Analysis and Mining, 2023
- Detecting fake news by exploring the consistency of multimodal dataInformation Processing & Management, 2021
- Multiple features based approach for automatic fake news detection on social networks using deep learningApplied Soft Computing, 2021
- Behind the cues: A benchmarking study for fake news detectionExpert Systems with Applications, 2019