Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection
Authors:
- Linmei HuLHLinmei Hu
- Ziwang ZhaoZZZiwang Zhao
- Weijian QiWQWeijian Qi
- Xuemeng SongXSXuemeng Song
- Liqiang NieLNLiqiang Nie
Preprint
Abstract
Fake news often involves multimedia information such as text and image to mislead readers, proliferating and expanding its influence. Most existing fake news detection methods apply the co-attention mechanism to fuse multimodal features while ignoring the consistency of image and text in co-attention. In this paper, we propose multimodal matching-aware co-attention networks with mutual knowledge distillation for improving fake news detection. Specifically, we design an image-text matching-aware co-attention mechanism which captures the alignment of image and text for better multimodal fusion. The image-text matching representation can be obtained via a vision-language pre-trained model. Additionally, based on the designed image-text matching-aware co-attention mechanism, we propose to build two co-attention networks respectively centered on text and image for mutual knowledge distillation to improve fake news detection. Extensive experiments on three benchmark datasets demonstrate that our proposed model achieves state-of-the-art performance on multimodal fake news detection.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Hybrid CNN-Transformer Features for Visual Place RecognitionIEEE Transactions on Circuits and Systems for Video Technology, 2022
- VDTR: Video Deblurring With TransformerIEEE Transactions on Circuits and Systems for Video Technology, 2022
- AM³Net: Adaptive Mutual-Learning-Based Multimodal Data Fusion NetworkIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Dual Mutual Learning for Cross-Modality Person Re-IdentificationIEEE Transactions on Circuits and Systems for Video Technology, 2022
- A Transformer-Based Feature Segmentation and Region Alignment Method for UAV-View Geo-LocalizationIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Category-controlled Encoder-Decoder for Fake News DetectionIEEE Transactions on Knowledge and Data Engineering, 2021
- Mutual Learning and Feature Fusion Siamese Networks for Visual Object TrackingIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Detecting fake news by exploring the consistency of multimodal dataInformation Processing & Management, 2021
- An Integrated Multi-Task Model for Fake News DetectionIEEE Transactions on Knowledge and Data Engineering, 2021
- Novel Visual and Statistical Image Features for Microblogs News VerificationIEEE Transactions on Multimedia, 2016