Ensembles of Neural Networks for Robust Reinforcement Learning
Authors:
- Alexander HansAHAlexander Hans
- Steffen UdluftSUSteffen Udluft
- 1 December 2010
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 14, 401-406
Abstract
Reinforcement learning algorithms that employ neural networks as function approximators have proven to be powerful tools for solving optimal control problems. However, their training and the validation of final policies can be cumbersome as neural networks can suffer from problems like local minima or over fitting. When using iterative methods, such as neural fitted Q-iteration, the problem becomes even more pronounced since the network has to be trained multiple times and the training process in one iteration builds on the network trained in the previous iteration. Therefore errors can accumulate. In this paper we propose to use ensembles of networks to make the learning process more robust and produce near-optimal policies more reliably. We name various ways of combining single networks to an ensemble that results in a final ensemble policy and show the potential of the approach using a benchmark application. Our experiments indicate that majority voting is superior to Q-averaging and using heterogeneous ensembles (different network topologies) is advisable.Keywords
This publication has 7 references indexed in Scilit:
- Ensemble Algorithms in Reinforcement LearningIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- A Recurrent Control Neural Network for Data Efficient Reinforcement Learning2007
- Modeling Large Dynamical Systems with Dynamical Consistent Neural Networks2006
- Random ForestsMachine Learning, 2001
- Ensemble Methods in Machine Learning2000
- Bagging predictorsMachine Learning, 1996
- Stable Function Approximation in Dynamic Programming1995