Efficient Feature Selection and Multiclass Classification with Integrated Instance and Model Based Learning
Authors:
Open Access
- 1 January 2012
- journal article
- research article
- Published by SAGE Publications in Evolutionary Bioinformatics
- Vol. 8, 197-205
Abstract
Multiclass classification and feature (variable) selections are commonly encountered in many biological and medical applications. However, extending binary classification approaches to multiclass problems is not trivial. Instance-based methods such as the K nearest neighbor (KNN) can naturally extend to multiclass problems and usually perform well with unbalanced data, but suffer from the curse of dimensionality. Their performance is degraded when applied to high dimensional data. On the other hand, model-based methods such as logistic regression require the decomposition of the multiclass problem into several binary problems with one-vs.-one or one-vs.-rest schemes. Even though they can be applied to high dimensional data with L1 or L p penalized methods, such approaches can only select independent features and the features selected with different binary problems are usually different. They also produce unbalanced classification problems with one vs. the rest scheme even if the original multiclass problem is balanced. By combining instance-based and model-based learning, we propose an efficient learning method with integrated KNN and constrained logistic regression (KNNLog) for simultaneous multiclass classification and feature selection. Our proposed method simultaneously minimizes the intra-class distance and maximizes the interclass distance with fewer estimated parameters. It is very efficient for problems with small sample size and unbalanced classes, a case common in many real applications. In addition, our model-based feature selection methods can identify highly correlated features simultaneously avoiding the multiplicity problem due to multiple tests. The proposed method is evaluated with simulation and real data including one unbalanced microRNA dataset for leukemia and one multi-class metagenomic dataset from the Human Microbiome Project (HMP). It performs well with limited computational experiments.Keywords
This publication has 20 references indexed in Scilit:
- Sparse distance-based learning for simultaneous multiclass classification and feature selection of metagenomic dataBioinformatics, 2011
- miR-181b is a biomarker of disease progression in chronic lymphocytic leukemiaBlood, 2011
- MicroRNA miR-125b causes leukemiaProceedings of the National Academy of Sciences, 2010
- Bacterial Community Variation in Human Body Habitats Across Space and TimeScience, 2009
- Introducing mothur: Open-Source, Platform-Independent, Community-Supported Software for Describing and Comparing Microbial CommunitiesApplied and Environmental Microbiology, 2009
- Some sharp performance bounds for least squares regression with L1 regularizationThe Annals of Statistics, 2009
- MicroRNA expression profiling of human metastatic cancers identifies cancer gene targetsThe Journal of Pathology, 2009
- The sparsity and bias of the Lasso selection in high-dimensional linear regressionThe Annals of Statistics, 2008
- Distinctive microRNA signature of acute myeloid leukemia bearing cytoplasmic mutated nucleophosminProceedings of the National Academy of Sciences, 2008
- Sparse Logistic Regression with Lp Penalty for Biomarker IdentificationStatistical Applications in Genetics and Molecular Biology, 2007