ETMA: Efficient Transformer-Based Multilevel Attention Framework for Multimodal Fake News Detection
- 20 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 11 (4) , 5015-5027
Abstract
In this new digital era, social media has created a severe impact on the lives of people. In recent times, fake news content on social media has become one of the major challenging problems for society. The dissemination of fabricated and false news articles includes multimodal data in the form of text and images. The previous methods have mainly focused on unimodal analysis. Moreover, for multimodal analysis, researchers fail to keep the unique characteristics corresponding to each modality. This article aims to overcome these limitations by proposing an efficient transformer-based multilevel attention (ETMA) framework for multimodal fake news detection, which comprises the following components: a visual attention-based encoder, a textual attention-based encoder, and joint attention-based learning. Each component utilizes different forms of attention mechanisms and uniquely deals with multimodal data to detect fraudulent content. The efficacy of the proposed network is validated by conducting several experiments on four real-world fake news datasets: Twitter, Jruvika fake news dataset, Pontes fake news dataset, and Risdal fake news dataset using multiple evaluation metrics. The results show that the proposed method outperforms the baseline methods on all four datasets. Furthermore, the computation time of the model is also lower than the state-of-the-art methods.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networksInformation Processing & Management, 2021
- Two-Path Deep Semisupervised Learning for Timely Fake News DetectionIEEE Transactions on Computational Social Systems, 2020
- Reasoning on the Relation: Enhancing Visual Representation for Visual Question Answering and Cross-Modal RetrievalIEEE Transactions on Multimedia, 2020
- Uni-and-Bi-Directional Video Prediction via Learning Object-Centric TransformationIEEE Transactions on Multimedia, 2019
- MVAE: Multimodal Variational Autoencoder for Fake News Detection2019
- Multi-Turn Video Question Answering via Hierarchical Attention Context Reinforced NetworksIEEE Transactions on Image Processing, 2019
- Attention CoupleNet: Fully Convolutional Attention Coupling Network for Object DetectionIEEE Transactions on Image Processing, 2018
- Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs2017
- Deep Reinforcement Learning With Visual Attention for Vehicle ClassificationIEEE Transactions on Cognitive and Developmental Systems, 2016
- VQA: Visual Question Answering2015