Two Timescale Analysis of the Alopex Algorithm for Optimization
Authors:
- 1 November 2002
- journal article
- Published by MIT Press in Neural Computation
- Vol. 14 (11) , 2729-2750
Abstract
Alopex is a correlation-based gradient-free optimization technique useful in many learning problems. However, there are no analytical results on the asymptotic behavior of this algorithm. This article presents a new version of Alopex that can be analyzed using techniques of two timescale stochastic approximation method. It is shown that the algorithm asymptotically behaves like a gradient-descent method, though it does not need (or estimate) any gradient information. It is also shown, through simulations, that the algorithm is quite effective.Keywords
This publication has 17 references indexed in Scilit:
- ALOPEX-B: A NEW, SIMPLER, BUT YET FASTER VERSION OF THE ALOPEX TRAINING ALGORITHMInternational Journal of Neural Systems, 2001
- Stochastic approximation algorithms: Overview and recent trendsSādhanā, 1999
- Stochastic approximation with two time scalesSystems & Control Letters, 1997
- Multivariate decision treesMachine Learning, 1995
- A System for Induction of Oblique Decision TreesJournal of Artificial Intelligence Research, 1994
- On the handling of continuous-valued attributes in decision tree generationMachine Learning, 1992
- Weight Perturbation: An Optimal Architecture and Learning Technique for Analog VLSI Feedforward and Recurrent Multilayer NetworksNeural Computation, 1991
- Model-free distributed learningIEEE Transactions on Neural Networks, 1990
- Brainstem control of sensory information: A mechanism for perceptionInternational Journal of Psychophysiology, 1985
- ALOPEX: A stochastic method for determining visual receptive fieldsVision Research, 1974