Abstract
The significance of multimodal feature fusion in multimodal-based fake news detection is crucial. However, each modality’s contribution to the final detection outcome varies, and due to substantial differences in data distribution, a simple fusion approach is often inadequate. While existing works attempt to perform multimodal fusion using attention mechanisms, they still face the challenge of modal laziness—where a modality may be underutilized during training due to its low weight, resulting in insufficient learning. In this article, we propose the model alternative learning network (AlLNet). AlLNet addresses the issue of modal laziness by learning unimodal features separately during the training stage and employing an alternating independent training process. Additionally, it incorporates contrast learning to enhance the discriminative ability of image features. During the inference stage, AlLNet evaluates the uncertainty of different modalities using entropy and combines their predictions dynamically with appropriate weights to produce the final result. Moreover, AlLNet is designed to be lightweight and effectively handles modalities missing, making it well-suited for real-world applications. Our experiments conducted on three real-world datasets in English and Chinese demonstrate the advantages and effectiveness of our approach.
Funding Information
  • Key Research and Development Program in Shaanxi Province of China (2024GX-YBXM-124)

This publication has 22 references indexed in Scilit: