Identifying multimodal misinformation leveraging novelty detection and emotion recognition
- 6 June 2023
- journal article
- research article
- Published by Springer Nature in Journal of Intelligent Information Systems
- Vol. 61 (3) , 673-694
Abstract
With the growing presence of multimodal content on the web, a specific category of fake news is rampant on popular social media outlets. In this category of fake online information, real multimedia contents (images, videos) are used in different but related contexts with manipulated texts to mislead the readers. The presence of seemingly non-manipulated multimedia content reinforces the belief in the associated fabricated textual content. Detecting this category of misleading multimedia fake news is almost impossible without relevance to any prior knowledge. In addition to this, the presence of highly novel and emotion-invoking contents can fuel the rapid dissemination of such fake news. To counter this problem, in this paper, we first introduce a novel multimodal fake news dataset that includes background knowledge (from authenticate sources) of the misleading articles. Second, we design a multimodal framework using Supervised Contrastive Learning (SCL) based novelty detection and Emotion Prediction tasks for fake news detection. We perform extensive experiments to reveal that our proposed model outperforms the state-of-the-art (SOTA) models.Keywords
This publication has 39 references indexed in Scilit:
- Transfer learning using VGG-16 with Deep Convolutional Neural Network for Classifying ImagesInternational Journal of Scientific and Research Publications, 2019
- SpotFake: A Multi-modal Framework for Fake News Detection2019
- MVAE: Multimodal Variational Autoencoder for Fake News Detection2019
- Fact-Checking Meets Fauxtography: Verifying Claims About Images2019
- Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias2018
- EANN2018
- The spread of true and false news onlineScience, 2018
- Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs2017
- Affective image classification using features inspired by psychology and art theory2010
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960