Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection

Authors:
  • LHLinmei Hu
    Linmei Hu
  • ZZZiwang Zhao
    Ziwang Zhao
  • WQWeijian Qi
    Weijian Qi
  • XSXuemeng Song
    Xuemeng Song
  • LNLiqiang Nie
    Liqiang Nie

Abstract
Fake news often involves multimedia information such as text and image to mislead readers, proliferating and expanding its influence. Most existing fake news detection methods apply the co-attention mechanism to fuse multimodal features while ignoring the consistency of image and text in co-attention. In this paper, we propose multimodal matching-aware co-attention networks with mutual knowledge distillation for improving fake news detection. Specifically, we design an image-text matching-aware co-attention mechanism which captures the alignment of image and text for better multimodal fusion. The image-text matching representation can be obtained via a vision-language pre-trained model. Additionally, based on the designed image-text matching-aware co-attention mechanism, we propose to build two co-attention networks respectively centered on text and image for mutual knowledge distillation to improve fake news detection. Extensive experiments on three benchmark datasets demonstrate that our proposed model achieves state-of-the-art performance on multimodal fake news detection.

This publication has 12 references indexed in Scilit: