Comparing Fine-Tuned RoBERTa with Traditional Machine Learning Models for Stance Detection in Political Tweets
Open Access
- 25 May 2025
- journal article
- Published by Institute of Central Computation and Knowledge Inc. in ICCK Transactions on Advanced Computing and Systems
- Vol. 1 (2) , 78-96
Abstract
Stance detection identifies a text’s position or attitude toward a given subject. A major challenge in Roman Urdu is the lack of a publicly available dataset for political stance detection. To address this gap, we constructed a high-quality dataset of 8,374 political tweets and comments using the Twitter API, annotated with stance labels: agree, disagree, and unrelated. The dataset captures diverse political viewpoints and user interactions. For feature representation, we employed TF-IDF due to its effectiveness in handling high-dimensional, context-sensitive Roman Urdu text. Several machine learning classifiers were evaluated, with Random Forest achieving the highest accuracy of 95%. Additionally, we fine-tuned the transformer-based RoBERTa model, which outperformed traditional methods with 97% accuracy. Our results demonstrate the potential of combining machine learning and deep learning for stance detection in low-resource languages. This study not only introduces a novel dataset but also provides a robust evaluation of methods, highlighting the importance of modern AI techniques in processing informal and multilingual text data.Keywords
This publication has 16 references indexed in Scilit:
- Semantic Tagging for the Urdu Language: Annotated Corpus and Multi-Target Classification MethodsACM Transactions on Asian and Low-Resource Language Information Processing, 2023
- A systematic review of machine learning techniques for stance detection and its applicationsNeural Computing and Applications, 2023
- Stance detection for online public opinion awareness: An overviewInternational Journal of Intelligent Systems, 2022
- Stance detection on social media: State of the art and trendsInformation Processing & Management, 2021
- P-Stance: A Large Dataset for Stance Detection in Political Domain2021
- Early Detection of Rumours on Twitter via Stance Transfer Learning2020
- Stance Detection in Web and Social Media: A Comparative Study2019
- Detecting stance in kannada social media code-mixed text using sentence embedding2017
- Turing at SemEval-2017 Task 8: Sequential Approach to Rumour Stance Classification with Branch-LSTM2017
- That is your evidence?: Classifying stance in online political debateDecision Support Systems, 2012