MCFEND: A Multi-source Benchmark Dataset for Chinese Fake News Detection
Open Access
- 13 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 4018-4027
Abstract
The prevalence of fake news across various online sources has had a significant influence on the public. Existing Chinese fake news detection datasets are limited to news sourced solely from Weibo. However, fake news originating from multiple sources exhibits diversity in various aspects, including its content and social context. Methods trained on purely one single news source can hardly be applicable to real-world scenarios. Our pilot experiment demonstrates that the F1 score of the state-of-the-art method that learns from a large Chinese fake news detection dataset, Weibo-21, drops significantly from 0.943 to 0.470 when the test data is changed to multi-source news data, failing to identify more than one-third of the multi-source fake news. To address this limitation, we constructed the first multi-source benchmark dataset for Chinese fake news detection, termed MCFEND, which is composed of news we collected from diverse sources such as social platforms, messaging apps, and traditional online news outlets. Notably, such news has been fact-checked by 14 authoritative fact-checking agencies worldwide. In addition, various existing Chinese fake news detection methods are thoroughly evaluated on our proposed dataset in cross-source, multi-source, and unseen source ways. MCFEND, as a benchmark dataset, aims to advance Chinese fake news detection approaches in real-world scenarios.Keywords
All Related Versions
Funding Information
- Guangdong Basic and Applied Basic Research Foundation (2022A1515011583, 2023A1515011562)
- Startup Grant (Tier 1) for New Academics of Hong Kong Baptist University under Grant AY2020/21
- One-Off Tier 2 Start-Up Grant (2020/2021) of Hong Kong Baptist University (RC-OFSGT2/20-21/COMM/002)
- National Natural Science Foundation of China (62202402)
- Hong Kong Research Grants Council Early Career Scheme (22202423)
- Germany/Hong Kong Joint Research Scheme sponsored by the Research Grants Council of Hong Kong and the German Academic Exchange Service of Germany (G-HKBU203/22)
This publication has 36 references indexed in Scilit:
- Fake news, disinformation and misinformation in social media: a reviewSocial Network Analysis and Mining, 2023
- Deep learning for fake news detection: A comprehensive surveyAI Open, 2022
- CHECKED: Chinese COVID-19 fake news datasetSocial Network Analysis and Mining, 2021
- Early Detection of Fake News with Multi-source Weak Social Supervision2021
- FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social MediaBig Data, 2020
- False News On Social MediaACM SIGMOD Record, 2019
- A survey on fake news and rumour detection techniquesInformation Sciences, 2019
- Detection and visualization of misleading content on TwitterInternational Journal of Multimedia Information Retrieval, 2017
- Exploiting Context for Rumour Detection in Social Media2017
- Deep Residual Learning for Image Recognition2016