An Analysis of the EM Algorithm and Entropy-Like Proximal Point Methods
Author:
- 1 February 2004
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Mathematics of Operations Research
- Vol. 29 (1) , 27-44
Abstract
The EM algorithm is a popular method for maximum likelihood estimation from incomplete data. This method may be viewed as a proximal point method for maximizing the log-likelihood function using an integral form of the Kullback-Leibler distance function. Motivated by this interpretation, we consider a proximal point method using an integral form of entropy-like distance function. We give a convergence analysis of the resulting proximal point method in the case where the cluster points lie in the interior of the objective function domain. This result is applied to a normal/independent example and a Gaussian mixture example to establish convergence of the EM algorithm on these examples. Further convergence analysis of the method for maximization over an orthant is given in low dimensions. Sublinear convergence and schemes for accelerating convergence are also discussed.Keywords
This publication has 21 references indexed in Scilit:
- Local Convergence of the Proximal Point Algorithm and Multiplier Methods Without MonotonicityMathematics of Operations Research, 2002
- Interior Proximal and Multiplier Methods Based on Second Order Homogeneous KernelsMathematics of Operations Research, 1999
- Proximal Point Methods and Nonconvex OptimizationJournal of Global Optimization, 1998
- A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants1998
- Proximal Minimization Methods with Generalized Bregman FunctionsSIAM Journal on Control and Optimization, 1997
- Penalty/Barrier Multiplier Methods for Convex Programming ProblemsSIAM Journal on Optimization, 1997
- Nonlinear rescaling and proximal-like methods in convex optimizationMathematical Programming, 1997
- Convergence Rate Analysis of Nonquadratic Proximal Methods for Convex and Linear ProgrammingMathematics of Operations Research, 1995
- A regularized dual-based iterative method for a class of image reconstruction problemsInverse Problems, 1993
- Error bounds and convergence analysis of feasible descent methods: a general approachAnnals of Operations Research, 1993