Locally adaptive metric nearest-neighbor classification
Authors:
- Jing PengJPJing Peng
- D. GunopulosDGD. Gunopulos
- 7 November 2002
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 24 (9) , 1281-1285
Abstract
Nearest-neighbor classification assumes locally constant class conditional probabilities. This assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality. Severe bias can be introduced under these conditions when using the nearest-neighbor rule. We propose a locally adaptive nearest-neighbor classification method to try to minimize bias. We use a chi-squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations. Neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones. As a result, the class conditional probabilities are smoother in the modified neighborhoods, whereby better classification performance can be achieved. The efficacy of our method is validated and compared against other techniques using both simulated and real-world data.Keywords
This publication has 13 references indexed in Scilit:
- Locally Weighted LearningArtificial Intelligence Review, 1997
- Bagging predictorsMachine Learning, 1996
- Discriminant adaptive nearest neighbor classificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 1996
- Similarity Metric Learning for a Variable-Kernel ClassifierNeural Computation, 1995
- Local Learning AlgorithmsNeural Computation, 1992
- Discriminant Analysis and Statistical Pattern Recognition1992
- The multi-class metric problem in nearest neighbour discrimination rulesPattern Recognition, 1990
- Locally Weighted Regression: An Approach to Regression Analysis by Local FittingJournal of the American Statistical Association, 1988
- The optimal distance measure for nearest neighbor classificationIEEE Transactions on Information Theory, 1981
- Consistent Nonparametric RegressionThe Annals of Statistics, 1977