A Comparison of Automatic Labelling Approaches for Sentiment Analysis
Preprint
Abstract
Labelling a large quantity of social media data for the task of supervised machine learning is not only time-consuming but also difficult and expensive. On the other hand, the accuracy of supervised machine learning models is strongly related to the quality of the labelled data on which they train, and automatic sentiment labelling techniques could reduce the time and cost of human labelling. We have compared three automatic sentiment labelling techniques: TextBlob, Vader, and Afinn to assign sentiments to tweets without any human assistance. We compare three scenarios: one uses training and testing datasets with existing ground truth labels; the second experiment uses automatic labels as training and testing datasets; and the third experiment uses three automatic labelling techniques to label the training dataset and uses the ground truth labels for testing. The experiments were evaluated on two Twitter datasets: SemEval-2013 (DS-1) and SemEval-2016 (DS-2). Results show that the Afinn labelling technique obtains the highest accuracy of 80.17% (DS-1) and 80.05% (DS-2) using a BiLSTM deep learning model. These findings imply that automatic text labelling could provide significant benefits, and suggest a feasible alternative to the time and cost of human labelling efforts.Keywords
All Related Versions
- Version 1, 2022-11-05, ArXiv
- Published version: , 312.
This publication has 10 references indexed in Scilit:
- Determining the Efficiency of Drugs Under Special Conditions From Users’ Reviews on Healthcare Web ForumsIEEE Access, 2021
- Sentiment Analysis of COVID-19 tweets by Deep Learning Classifiers—A study to show how popularity is affecting accuracy in social mediaApplied Soft Computing, 2020
- Twitter Sentiment Analysis Using Lexical or Rule Based Approach: A Case Study2020
- Sentiment Analysis of Comment Texts Based on BiLSTMIEEE Access, 2019
- Distinguishing between facts and opinions for sentiment analysis: Survey and challengesInformation Fusion, 2018
- Machine Learning-Based Sentiment Analysis for Twitter AccountsMathematical and Computational Applications, 2018
- Stance and Sentiment in TweetsACM Transactions on Internet Technology, 2017
- CNN for situations understanding based on sentiment analysis of twitter dataProcedia Computer Science, 2017
- Rating Prediction Based on Social Sentiment From Textual ReviewsIEEE Transactions on Multimedia, 2016
- Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction AlgorithmJournal of Artificial Intelligence Research, 1995