A cartpole experiment benchmark for trainable controllers
Authors:
- 1 October 1993
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems
- Vol. 13 (5) , 40-51
Abstract
The inverted pendulum problem, i.e., the cartpole, which is often used for demonstrating the success of neural network learning methods, is addressed. It is shown that a random search in weight space can quickly uncover coefficients (weights) for controllers that work over a wide range of initial conditions. This result indicates that success in finding a satisfactory neural controller is not sufficient proof for the effectiveness of unsupervised training methods. By analyzing the dynamics of the linear controller, the cartpole problem is reformulated to make it a more stringent test for neural training methods. A review of the literature on unsupervised training methods for cartpole controllers shows that the published results are difficult to compare and that for most of the methods there is not clear evidence of better performance than the random search method.Keywords
This publication has 15 references indexed in Scilit:
- Neuromorphic learning of continuous-valued mappings from noise-corrupted dataIEEE Transactions on Neural Networks, 1991
- USE OF CMAC NEURAL NETWORKS IN REINFORCEMENT SELF-LEARNING CONTROL11The work was supported by the National Science Foundation under the grant MIP-8808995.1991
- GENETICALLY PROGRAMMED NEURAL NETWORK FOR SOLVING POLE-BALANCING PROBLEM1991
- CMAC-based adaptive critic self-learning controlIEEE Transactions on Neural Networks, 1991
- A neurocontroller based on model feedback and the adaptive heuristic critic1990
- A self-organizing controller for dynamic processes using neural networks1990
- Unsupervising adaption neural-network control1990
- An adaptive 'broom balancer' with visual inputs1988
- Neuronlike adaptive elements that can solve difficult learning control problemsIEEE Transactions on Systems, Man, and Cybernetics, 1983
- Punish/Reward: Learning with a Critic in Adaptive Threshold SystemsIEEE Transactions on Systems, Man, and Cybernetics, 1973