The Impact of Data Preparation and Model Complexity on the Natural Language Classification of Chinese News Headlines
Open Access
- 22 March 2024
- journal article
- research article
- Published by MDPI AG in Algorithms
- Vol. 17 (4) , 132
Abstract
Given the emergence of China as a political and economic power in the 21st century, there is increased interest in analyzing Chinese news articles to better understand developing trends in China. Because of the volume of the material, automating the categorization of Chinese-language news articles by headline text or titles can be an effective way to sort the articles into categories for efficient review. A 383,000-headline dataset labeled with 15 categories from the Toutiao website was evaluated via natural language processing to predict topic categories. The influence of six data preparation variations on the predictive accuracy of four algorithms was studied. The simplest model (Naïve Bayes) achieved 85.1% accuracy on a holdout dataset, while the most complex model (Neural Network using BERT) demonstrated 89.3% accuracy. The most useful data preparation steps were identified, and another goal examined the underlying complexity and computational costs of automating the categorization process. It was discovered the BERT model required 170x more time to train, was slower to predict by a factor of 18,600, and required 27x more disk space to save, indicating it may be the best choice for low-volume applications when the highest accuracy is needed. However, for larger-scale operations where a slight performance degradation is tolerated, the Naïve Bayes algorithm could be the best choice. Nearly one in four records in the Toutiao dataset are duplicates, and this is the first published analysis with duplicates removed.Keywords
This publication has 17 references indexed in Scilit:
- Adapting Feature Selection Algorithms for the Classification of Chinese TextsSystems, 2023
- Short Text Classification of Chinese with Label Information AssistingACM Transactions on Asian and Low-Resource Language Information Processing, 2023
- Effective short text classification via the fusion of hybrid features for IoT social dataDigital Communications and Networks, 2022
- Chinese Short Text Classification by ERNIE Based on LTC_BlockWireless Communications and Mobile Computing, 2022
- Efficient Implementation using Multinomial Naive Bayes for Prediction of Fake Job ProfileInternational Journal of Advanced Research in Science, Communication and Technology, 2021
- A Text classification algorithm based on topic model and convolutional neural networkJournal of Physics: Conference Series, 2021
- Feature-Enhanced Nonequilibrium Bidirectional Long Short-Term Memory Model for Chinese Text ClassificationIEEE Access, 2020
- Chinese Document Classification with Bi-directional Convolutional Language Model2020
- An Introduction to Statistical Learning2013
- A systematic analysis of performance measures for classification tasksInformation Processing & Management, 2009