Reinforcement Learning: A Survey
Preprint
Abstract
This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Rapid, safe, and incremental learning of navigation strategiesIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1996
- Robot shaping: developing autonomous agents through learningArtificial Intelligence, 1994
- Associative Reinforcement Learning: Functions in k-DNFMachine Learning, 1994
- TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level PlayNeural Computation, 1994
- Robot juggling: implementation of memory-based learningIEEE Control Systems, 1994
- A survey of algorithmic methods for partially observed Markov decision processesAnnals of Operations Research, 1991
- A self-learning rule-based controller employing approximate reasoning and neural net conceptsInternational Journal of Intelligent Systems, 1991
- A stochastic reinforcement learning algorithm for learning real-valued functionsNeural Networks, 1990
- Optimum Systems ControlIEEE Transactions on Systems, Man, and Cybernetics, 1979
- Learning Automata - A SurveyIEEE Transactions on Systems, Man, and Cybernetics, 1974