Hybrid Dimensionality Reduction Forest With Pruning for High-Dimensional Data Classification
Open Access
- 24 February 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 40138-40150
Abstract
The classification of high-dimensional data is a challenge in machine learning. Traditional classifier ensemble methods improve the diversity of classifiers through either dimensionality reduction or sample selection for high-dimensional data classification. However, these methods have several limitations: 1) dimensionality reduction methods easily cause information loss, which leads to a decrease in accuracy; 2) sample selection methods are susceptible to noise and redundant features. To address the above limitations, we propose a novel hybrid dimensionality reduction forest (HDRF) to increase the diversity of an integrated system from feature space and sample space. First, a tree-based feature selection algorithm is employed to partition effective features. Then the Bagging method is applied to obtain diverse training subsets. To fully retain and mine the important information of the unselected samples, a sample-feature based transformation process (SFTP) is proposed to generate the extended features. Since PCA can effectively reduce dimension and remove noise features, it is applied to compress the unselected features and the extended features into the new features which are compact and compensatory. Further, a novel classifier ensemble pruning framework (HDRFPF) based on HDRF is designed to remove redundant and invalid classifiers. Experimental results on 23 high-dimensional data sets verify that our method outperforms mainstream classifier ensemble methods.Keywords
Funding Information
- National Natural Science Foundation of China (61751205, 61722205, 61572199, 61572540, U1611461)
- Key Research and Development Program of Guangdong Province (2018B010107002)
- Natural Science Foundation of Guangdong Province (2016A030308013)
This publication has 60 references indexed in Scilit:
- Feature Selection Inspired Classifier Ensemble ReductionIEEE Transactions on Cybernetics, 2013
- Clustering cancer gene expression data: a comparative studyBMC Bioinformatics, 2008
- NeC4.5: neural ensemble based C4.5IEEE Transactions on Knowledge and Data Engineering, 2004
- Ensembling neural networks: Many could be better than allArtificial Intelligence, 2002
- Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclassesProceedings of the National Academy of Sciences, 2001
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001
- Distinct types of diffuse large B-cell lymphoma identified by gene expression profilingNature, 2000
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997
- Bagging predictorsMachine Learning, 1996