A new algorithm for stochastic optimization
Author:
- 4 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 364-366
Abstract
Classical stochastic optimization algorithms have severe problems associated with them: they converge extremely slowly on problems where the objective function is very flat, and they often diverge when the objective function is steep. The author has developed a stochastic optimization algorithm that is more robust than the older algorithms in that it is guaranteed to converge on a larger class of problems. This algorithm is guaranteed to converge even when the iterates are not assumed a priori to be bounded. This algorithm is also observed to converge faster on a significant class of problems. As the parameters can be chosen so that the new algorithm behaves very much like the older algorithms (except that it converges on a larger class of problems), this algorithm should always be used in preference to the older algorithms.Keywords
This publication has 3 references indexed in Scilit:
- Accelerated Stochastic ApproximationThe Annals of Mathematical Statistics, 1958
- Stochastic Estimation of the Maximum of a Regression FunctionThe Annals of Mathematical Statistics, 1952
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951