Improving Probabilistic Models In Text Classification Via Active Learning
Open Access
- 5 August 2024
- journal article
- research article
- Published by Cambridge University Press (CUP) in American Political Science Review
- Vol. 119 (2) , 985-1002
Abstract
Social scientists often classify text documents to use the resulting labels as an outcome or a predictor in empirical research. Automated text classification has become a standard tool since it requires less human coding. However, scholars still need many human-labeled documents for training. To reduce labeling costs, we propose a new algorithm for text classification that combines a probabilistic model with active learning. The probabilistic model uses both labeled and unlabeled data, and active learning concentrates labeling efforts on difficult documents to classify. Our validation study shows that with few labeled data, the classification performance of our algorithm is comparable to state-of-the-art methods at a fraction of the computational cost. We replicate the results of two published articles with only a small fraction of the original labeled data used in those studies and provide open-source software to implement our method.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Machine Learning Predictions as Regression CovariatesPolitical Analysis, 2020
- Active Learning Approaches for Labeling Text: Review and Assessment of the Performance of Active Learning ApproachesPolitical Analysis, 2020
- Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information EffectsPolitical Analysis, 2018
- Estimating Spatial Preferences from Votes and TextPolitical Analysis, 2018
- Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster SystemsPolitical Analysis, 2018
- Deadly Clerics2017
- Electoral Reform and National Security in Japan2016
- Multi-Class Active Learning by Uncertainty Sampling with Diversity MaximizationInternational Journal of Computer Vision, 2014
- Two faces of active learningTheoretical Computer Science, 2011
- The Elements of Statistical Learning2009