A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit
Authors:
- 28 February 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 696-701
Abstract
In this paper, we propose an optimized Transformer model that integrates Bayesian algorithms with a Bidirectional Gated Recurrent Unit (BiGRU), and apply it to fake news classification for the first time. First, we employ the TF-IDF method to extract features from news texts and transform them into numeric representations to facilitate subsequent machine learning tasks. Two sets of experiments are then conducted for fake news detection and classification: one using a Transformer model optimized only with BiGRU, and the other incorporating Bayesian algorithms into the BiGRU-based Transformer. Experimental results show that the BiGRU-optimized Transformer achieves 100% accuracy on the training set and 99.67% on the test set, while the addition of the Bayesian algorithm maintains 100% accuracy on the training set and slightly improves test-set accuracy to 99.73%. This indicates that the Bayesian algorithm boosts model accuracy by 0.06%, further enhancing the detection capability for fake news. Moreover, the proposed algorithm converges rapidly at around the 10th training epoch with accuracy nearing 100%, demonstrating both its effectiveness and its fast classification ability. Overall, the optimized Transformer model, enhanced by the Bayesian algorithm and BiGRU, exhibits excellent continuous learning and detection performance, offering a robust technical means to combat the spread of fake news in the current era of information overload.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Improving Academic Skills Assessment with NLP and Ensemble Learning2024
- Analysis and Classification of Fake News Using Sequential Pattern MiningBig Data Mining and Analytics, 2024
- GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost2023
- Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance2023
- Fake News Classification using transformer based enhanced LSTM and BERTInternational Journal of Cognitive Computing in Engineering, 2022
- A Taxonomy of Fake News Classification Techniques: Survey and Implementation AspectsIEEE Access, 2022
- Machine learning for fake news classification with optimal feature selectionSoft Computing, 2022
- Fake News Classification Based on Content Level FeaturesApplied Sciences, 2022
- Toward a Better Performance Evaluation Framework for Fake News ClassificationProceedings of the International AAAI Conference on Web and Social Media, 2020