Text Data Augmentation Techniques for Word Embeddings in Fake News Classification
Open Access
- 26 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 31538-31550
Abstract
Contemporary language models heavily rely on large corpora for their training. The larger the corpus, the better a model can capture various semantic relationships. The issue at hand appears to be the limited scope of the corpora used. One potential solution to this problem is the application of data augmentation techniques to expand the existing corpus. Data augmentation encompasses several techniques for corpus augmentation. In this article, we delve deeper into the analysis of three techniques: Synonym Replacement, Back Translation, and Reduction of Function Words. Utilizing these three techniques, we prepared diverse versions of the corpus employed for training Word2Vec Skip-gram models. These techniques were validated through extrinsic evaluation, wherein Word2Vec Skip-gram models were used to generate word vectors for classifying fake news articles. Performance measures of the generated classifiers were analyzed. The study highlights significant statistically differences in classifier outcomes between augmented and original corpora. Specifically, Back Translation significantly enhances accuracy, notably with Support Vector and Bernoulli Naive Bayes models. Conversely, Reduction of Function Words (FWD) improves Logistic Regression, while the original corpus excels in Random Forest classification. The article also includes an intrinsic evaluation involving lexical semantic relations between word pairs. The intrinsic evaluation highlights nuanced differences in semantic relations across augmented corpora. Notably, the Back Translation (BT) corpus better aligns with established lexical resources, showcasing promising improvements in understanding specific semantic relationships.Keywords
Funding Information
- Scientific Grant Agency of the Ministry of Education of Slovak Republic and Slovak Academy of Sciences (VEGA-1/0734/24)
- European Commission ERASMUS+ Program 2021 (2021-1-SK01-KA220-HED-000032095)
This publication has 22 references indexed in Scilit:
- Optimizing fake news detection for Arabic context: A multitask learning approach with transformers and an enhanced Nutcracker Optimization AlgorithmKnowledge-Based Systems, 2023
- On the use of text augmentation for stance and fake news detectionJournal of Information and Telecommunication, 2023
- Multimodal fake news detection through data augmentation-based contrastive learningApplied Soft Computing, 2023
- Data augmentation techniques in natural language processingApplied Soft Computing, 2022
- Augmentation-Based Ensemble Learning for Stance and Fake News Detection2022
- Valuating requirements arguments in the online user's forum for requirements decision‐making: The CrowdRE‐VArg frameworkSoftware: Practice and Experience, 2022
- Toward the Development of Large-Scale Word Embedding for Low-Resourced LanguageIEEE Access, 2022
- Text data augmentations: Permutation, antonyms and negationExpert Systems with Applications, 2021
- WELFake: Word Embedding Over Linguistic Features for Fake News DetectionIEEE Transactions on Computational Social Systems, 2021
- Intrinsic or Extrinsic Evaluation: An Overview of Word Embedding Evaluation2018