Overfitting Reduction of Text Classification Based on AdaBELM
Open Access
Abstract
Overfitting is an important problem in machine learning. Several algorithms, such as the extreme learning machine (ELM), suffer from this issue when facing high-dimensional sparse data, e.g., in text classification. One common issue is that the extent of overfitting is not well quantified. In this paper, we propose a quantitative measure of overfitting referred to as the rate of overfitting (RO) and a novel model, named AdaBELM, to reduce the overfitting. With RO, the overfitting problem can be quantitatively measured and identified. The newly proposed model can achieve high performance on multi-class text classification. To evaluate the generalizability of the new model, we designed experiments based on three datasets, i.e., the 20 Newsgroups, Reuters-21578, and BioMed corpora, which represent balanced, unbalanced, and real application data, respectively. Experiment results demonstrate that AdaBELM can reduce overfitting and outperform classical ELM, decision tree, random forests, and AdaBoost on all three text-classification datasets; for example, it can achieve 62.2% higher accuracy than ELM. Therefore, the proposed model has a good generalizability.Keywords
Funding Information
- National Natural Science Foundation of China (61602207, 61572228, 61472158, 61373050)
- Science Technology Development Project from Jilin Province (20160101247JC)
- Zhuhai Premier Discipline Enhancement Scheme
- Guangdong Premier Key-Discipline Enhancement Scheme
- the Educational Commission of Jilin Province
- China Scholarship Council
This publication has 35 references indexed in Scilit:
- Ensemble delta test-extreme learning machine (DT-ELM) for regressionNeurocomputing, 2013
- On the doubt about margin explanation of boostingArtificial Intelligence, 2013
- Optimizing extreme learning machines via ridge regression and batch intrinsic plasticityNeurocomputing, 2013
- Extreme learning machines: a surveyInternational Journal of Machine Learning and Cybernetics, 2011
- Realtime training on mobile devices for face recognition applicationsPattern Recognition, 2010
- Exploration–exploitation tradeoff using variance estimates in multi-armed banditsTheoretical Computer Science, 2009
- A fast pruned-extreme learning machine for classification problemNeurocomputing, 2008
- Top 10 algorithms in data miningKnowledge and Information Systems, 2007
- Machine learning in automated text categorizationACM Computing Surveys, 2002
- The strength of weak learnabilityMachine Learning, 1990