AMONet: Alternating Modality Optimization Network for Multimodal Rumor Detection
- 5 June 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 11387-11400
Abstract
The rapid development of social media provides diversified channels for the dissemination of rumors. How to accurately identify rumor information under different modalities becomes a significant challenge in the current field of information security. Limited by the heterogeneity of multimodal data itself, existing multimodal rumor detection methods often face the issue of modality laziness, that is, the model converges prematurely during the training process, resulting in some modality encoders failing to fully learn effective representations, ultimately weakening the model’s detection performance. To alleviate the problem of modality laziness, we propose an innovative alternating modality optimization network (AMONet), whose core innovations are embodied in three aspects: (1) An alternating modality optimization (AMO) parameter updating mechanism is proposed, to balance optimization opportunities for each modality encoder by cyclically training the branch networks of each modality; (2) To tackle the knowledge forgetting problem caused by alternating optimization, a fisher information-based regularization (FIR) method is introduced, which can impose elastic constraints on specific parameters to preserve crucial knowledge; (3) An innovative quantitative evaluation metric, modality inconsistency variance (MIV), is proposed to characterize modality laziness. Extensive experiments demonstrate that AMONet outperforms previous benchmarks.Keywords
Funding Information
- National Natural Science Foundation of China (62273101)
- National Natural Science Foundation of China (62141606)
- Guangdong Basic Research Program and the Basic and Applied Basic Research (2024A1515240069)
This publication has 44 references indexed in Scilit:
- Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction2025
- A Survey of Approaches to Early Rumor Detection on Microblogging Platforms: Computational and Socio‐Psychological InsightsWIREs Data Mining and Knowledge Discovery, 2025
- OmniFuse: A general modality fusion framework for multi-modality learning on low-quality medical dataInformation Fusion, 2024
- MIGCL: Fake news detection with multimodal interaction and graph contrastive learning networksApplied Intelligence, 2024
- MSER: Multimodal speech emotion recognition using cross-attention with deep fusionExpert Systems with Applications, 2023
- Hierarchical graph attention networks for multi-modal rumor detection on social mediaNeurocomputing, 2023
- A survey on rumor detection and prevention in social media using deep learningKnowledge and Information Systems, 2023
- Multimodal fake news detection via progressive fusion networksInformation Processing & Management, 2022
- Optimization and fine-tuning of DenseNet model for classification of COVID-19 cases in medical imagingInternational Journal of Information Management Data Insights, 2021
- Rumor detection based on propagation graph neural network with attention mechanismExpert Systems with Applications, 2020