Multimodal false information detection method based on Text-CNN and SE module
Open Access
- 23 November 2022
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 17 (11) , e0277463
Abstract
False information detection can detect false information in social media and reduce its negative impact on society. With the development of multimedia, the multimodal content contained in false information is increasing, so it is important to use multimodal features to detect false information. This paper mainly uses information from two modalities, text and image. The features extracted by the backbone network are not further processed in the previous work, and the problems of noise and information loss in the process of fusing multimodal features are ignored. This paper proposes a false information detection method based on Text-CNN and SE modules. We use Text-CNN to process the text and image features extracted by BERT and Swin-transformer to enhance the quality of the features. In addition, we use the modified SE module to fuse text and image features and reduce the noise in the fusion process. Meanwhile, we draw on the idea of residual networks to reduce information loss in the fusion process by concatenating the original features with the fused features. Our model improves accuracy by 6.5% and 2.0% on the Weibo dataset and Twitter dataset compared to the attention based multimodal factorized bilinear pooling. The comparative experimental results show that the proposed model can improve the accuracy of false information detection. The results of ablation experiments further demonstrate the effectiveness of each module in our model.Keywords
Funding Information
- National Natural Science Foundation of China (62166042)
- National Natural Science Foundation of China (U2003207)
- Natural Science Foundation of Xinjiang, China (2021D01C076)
- Strengthening Plan of National Defense Science and Technology Foundation of China (2021-JCJQ-JJ-0059)
This publication has 28 references indexed in Scilit:
- MVAE: Multimodal Variational Autoencoder for Fake News Detection2019
- Multimodal Transformer for Unaligned Multimodal Language Sequences2019
- EANN2018
- Squeeze-and-Excitation Networks2018
- Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs2017
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking2017
- Credibility Assessment of Textual Claims on the Web2016
- Evaluating Event Credibility on Twitter2012
- Long Short-Term Memory2012