Bayesian adaptive nearest neighbor
- 24 February 2010
- journal article
- research article
- Published by Wiley in Statistical Analysis and Data Mining: An ASA Data Science Journal
- Vol. 3 (2) , 92-105
Abstract
Theknearest neighbor classification (k‐NN) is a very simple and popular method for classification. However, it suffers from a major drawback, it assumes constant local class posterior probability. It is also highly dependent on and sensitive to the choice of the number of neighborsk. In addition, it severely lacks the desired probabilistic formulation. In this article, we propose a Bayesian adaptive nearest neighbor method (BANN) that can adaptively select the shape of the neighborhood and the number of neighborsk. The shape of the neighborhood is automatically selected according to the concentration of the data around each query point with the help of discriminants. The neighborhood size is not predetermined and is kept free using a prior distribution. Thus, we are able to make the model to select the appropriate neighborhood size. The model is fitted using Markov Chain Monte Carlo (MCMC), so we are not using exactly one neighborhood size but a mixture ofk. Our BANN model is highly flexible, determining any local pattern in the data‐generating process, and adapting it to give an improved prediction. We have applied our model on four simulated data sets with special structures and five real‐life benchmark data sets. Our proposed BANN method demonstrates substantial improvement overk‐NN and discriminant adaptive nearest neighbor (DANN) in all nine case studies. It also outperforms the probabilistic nearest neighbor (PNN) in most of the data analyses. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 92‐105, 2010Keywords
This publication has 15 references indexed in Scilit:
- Simultaneous cancer classification and gene selection with Bayesian nearest neighbor method: An integrated approachComputational Statistics & Data Analysis, 2009
- Bayesian neural networks for bivariate binary data: an application to prostate cancer studyStatistics in Medicine, 2005
- Bayesian Classification of Tumours by Using Gene Expression DataJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- A Probabilistic Nearest Neighbour Method for Statistical Pattern RecognitionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2002
- Random ForestsMachine Learning, 2001
- Monte Carlo Statistical Methods1999
- Discriminant adaptive nearest neighbor classificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 1996
- Pattern Recognition and Neural Networks1996
- The Nature of Statistical Learning Theory1995
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1984