A comprehensive evaluation of oversampling techniques for enhancing text classification performance
Authors:
Top Cited Papers
Open Access
- 1 July 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-20
Abstract
Class imbalance is a common and critical challenge in text classification tasks, where the underrepresentation of certain classes often impairs the ability of classifiers to learn minority class patterns effectively. According to the “garbage in, garbage out” principle, even high-performing models may fail when trained on skewed distributions. To address this issue, this study investigates the impact of oversampling techniques, specifically the Synthetic Minority Over-sampling Technique (SMOTE) and thirty of its variants, on two benchmark text classification datasets: TREC and Emotions. Each dataset was vectorized using the MiniLMv2 transformer model to obtain semantically rich representations, and classification was performed using six machine learning algorithms. The balanced and imbalanced scenarios were compared in terms of F1-Score and Balanced Accuracy. This work constitutes, to the best of our knowledge, the first large-scale, systematic benchmarking of SMOTE-based oversampling methods in the context of transformer-embedded text classification. Furthermore, statistical significance of the observed performance differences was validated using the Friedman test. The results provide practical insights into the selection of oversampling techniques tailored to dataset characteristics and classifier sensitivity, supporting more robust and fair learning in imbalanced natural language processing tasks.Keywords
This publication has 57 references indexed in Scilit:
- The effect of rebalancing techniques on the classification performance in cyberbullying datasetsNeural Computing and Applications, 2023
- A study of machine learning-based models for detection, control, and mitigation of cyberbullying in online social mediaInternational Journal of Information Security, 2022
- A survey on sentiment analysis methods, applications, and challengesArtificial Intelligence Review, 2022
- LoRAS: an oversampling approach for imbalanced datasetsMachine Learning, 2020
- Feature selection for text classification: A reviewMultimedia Tools and Applications, 2018
- Learning from Imbalanced Data Sets2018
- DBSMOTE: Density-Based Synthetic Minority Over-sampling TEchniqueApplied Intelligence, 2011
- Safe-Level-SMOTE: Safe-Level-Synthetic Minority Over-Sampling TEchnique for Handling the Class Imbalanced Problem2009
- Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning2005
- SMOTEBoost: Improving Prediction of the Minority Class in Boosting2003