Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text
Authors:
Open Access
- 12 October 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 193248-193256
Abstract
Previous research on sentiment analysis mainly focuses on binary or ternary sentiment analysis in monolingual texts. However, in today’s social media such as micro-blogs, emotions are often expressed in bilingual or multilingual text called code-switching text, and people’s emotions are complex, including happiness, sadness, angry, afraid, surprise, etc. Different emotions may exist together, and the proportion of each emotion in the code-switching text is often unbalanced. Inspired by the recently proposed BERT model, we investigate how to fine-tune BERT for multi-label sentiment analysis in code-switching text in this paper. Our investigation includes the selection of pre-trained models and the fine-tuning methods of BERT on this task. To deal with the problem of the unbalanced distribution of emotions, a method based on data augmentation, undersampling and ensemble learning is proposed to get balanced samples and train different multi-label BERT classifiers. Our model combines the prediction of each classifier to get the final outputs. The experiment on the dataset of NLPCC 2018 shared task 1 shows the effectiveness of our model for the unbalanced code-switching text. The F1-Score of our model is higher than many previous models.Keywords
Funding Information
- National Key Research and Development Program of China (2018YFB1003801)
This publication has 16 references indexed in Scilit:
- Ensemble of Binary Classification for the Emotion Detection in Code-Switching Text2018
- Deep Contextualized Word Representations2018
- Emotion Detection in Code-switching Texts via Bilingual and Sentimental Information2015
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- Combining coregularization and consensus-based self-training for multilingual text categorization2010
- Exploratory Undersampling for Class-Imbalance LearningIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- SMOTE: Synthetic Minority Over-sampling TechniqueJournal of Artificial Intelligence Research, 2002
- The Art of Data AugmentationJournal of Computational and Graphical Statistics, 2001
- Back-Translation for Cross-Cultural ResearchJournal of Cross-Cultural Psychology, 1970