Stochastic Approximation and Newton’s Estimate of a Mixing Distribution
Authors:
- Ryan MartinRMRyan Martin
- Jayanta K. GhoshJGJayanta K. Ghosh
Open Access
- 1 August 2008
- journal article
- Published by Institute of Mathematical Statistics in Statistical Science
- Vol. 23 (3) , 365-382
Abstract
Many statistical problems involve mixture models and the need for computationally efficient methods to estimate the mixing distribution has increased dramatically in recent years. Newton [Sankhyā Ser. A 64 (2002) 306–322] proposed a fast recursive algorithm for estimating the mixing distribution, which we study as a special case of stochastic approximation (SA). We begin with a review of SA, some recent statistical applications, and the theory necessary for analysis of a SA algorithm, which includes Lyapunov functions and ODE stability theory. Then standard SA results are used to prove consistency of Newton’s estimate in the case of a finite mixture. We also propose a modification of Newton’s algorithm that allows for estimation of an additional unknown parameter in the model, and prove its consistency.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Consistency of a recursive estimate of mixing distributionsThe Annals of Statistics, 2009
- Equi-energy sampler with applications in statistical inference and statistical mechanicsThe Annals of Statistics, 2006
- Empirical bayes methods and false discovery rates for microarraysGenetic Epidemiology, 2002
- Computational Aspects of Nonparametric Bayesian Analysis with Applications to the Modeling of Multiple Binary SequencesJournal of Computational and Graphical Statistics, 2000
- Information-theoretic determination of minimax rates of convergenceThe Annals of Statistics, 1999
- A recursive algorithm for nonparametric analysis with missing dataBiometrika, 1999
- Nonparametric hierarchical Bayes via sequential imputationsThe Annals of Statistics, 1996
- A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation AlgorithmsJournal of the American Statistical Association, 1990
- Identifiability of Finite MixturesThe Annals of Mathematical Statistics, 1963
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951