Weighted Means in Stochastic Approximation of Minima
Authors:
- J. DipponJDJ. Dippon
- J. RenzJRJ. Renz
- 1 September 1997
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 35 (5) , 1811-1827
Abstract
Weighted averages of Kiefer--Wolfowitz-type procedures, which are driven by larger step lengths than usual, can achieve the optimal rate of convergence. A priori knowledge of a lower bound on the smallest eigenvalue of the Hessian matrix is avoided. The asymptotic mean squared error of the weighted averaging algorithm is the same as would emerge using a Newton-type adaptive algorithm. Several different gradient estimates are considered; one of them leads to a vanishing asymptotic bias. This gradient estimate applied with the weighted averaging algorithm usually yields a better asymptotic mean squared error than applied with the standard algorithm.Keywords
This publication has 15 references indexed in Scilit:
- On the Averaged Stochastic Approximation for Linear RegressionSIAM Journal on Control and Optimization, 1996
- An Optimum Design for Estimating the First DerivativeThe Annals of Statistics, 1995
- Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General ProcessesSIAM Journal on Control and Optimization, 1993
- Lower Rate of Convergence for Locating a Maximum of a FunctionThe Annals of Statistics, 1988
- Asymptotic behaviour of a class of stochastic approximation proceduresProbability Theory and Related Fields, 1986
- On Asymptotic Normality in Stochastic ApproximationThe Annals of Mathematical Statistics, 1968
- On the Choice of Design in Stochastic Approximation MethodsThe Annals of Mathematical Statistics, 1968
- Stochastic Approximation of Minima with Improved Asymptotic SpeedThe Annals of Mathematical Statistics, 1967
- Multidimensional Stochastic Approximation MethodsThe Annals of Mathematical Statistics, 1954
- Stochastic Estimation of the Maximum of a Regression FunctionThe Annals of Mathematical Statistics, 1952