Beyond Feedforward Models Trained by Backpropagation: A Practical Training Tool for a More Efficient Universal Approximator
Authors:
- Roman IlinRIRoman Ilin
- Robert KozmaRKRobert Kozma
- Paul J. WerbosPWPaul J. Werbos
- 20 May 2008
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 19 (6) , 929-937
Abstract
Cellular simultaneous recurrent neural network (SRN) has been shown to be a function approximator more powerful than the multilayer perceptron (MLP). This means that the complexity of MLP would be prohibitively large for some problems while SRN could realize the desired mapping with acceptable computational constraints. The speed of training of complex recurrent networks is crucial to their successful application. This work improves the previous results by training the network with extended Kalman filter (EKF). We implemented a generic cellular SRN (CSRN) and applied it for solving two challenging problems: 2-D maze navigation and a subset of the connectedness problem. The speed of convergence has been improved by several orders of magnitude in comparison with the earlier results in the case of maze navigation, and superior generalization has been demonstrated in the case of connectedness. The implications of this improvements are discussed.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Cellular SRN Trained by Extended Kalman Filter Shows Promise for ADP2006
- Quantum-Inspired Evolutionary Algorithms and Binary Particle Swarm Optimization for Training MLP and SRN Neural NetworksJournal of Computational and Theoretical Nanoscience, 2005
- A self-learning evolutionary chess programProceedings of the IEEE, 2004
- Evolving an expert checkers playing program without using human expertiseIEEE Transactions on Evolutionary Computation, 2001
- IMPLEMENTING BACK-PROPAGATION- THROUGH-TIME LEARNING ALGORITHM USING CELLULAR NEURAL NETWORKSInternational Journal of Bifurcation and Chaos, 1999
- Enhanced Multi-Stream Kalman Filter Training for Recurrent Networks1998
- Adaptive critic designsIEEE Transactions on Neural Networks, 1997
- Approximation and estimation bounds for artificial neural networksMachine Learning, 1994
- Consistency of HDP applied to a simple reinforcement learning problemNeural Networks, 1990
- Backpropagation through time: what it does and how to do itProceedings of the IEEE, 1990