Reinforcement Learning with Case-Based Heuristics for RoboCup Soccer Keepaway
Authors:
- Luiz A. Celiberto Jr.LJLuiz A. Celiberto Jr.
- Jackson P. MatsuuraJMJackson P. Matsuura
- Ramon Lopez de MantarasRMRamon Lopez de Mantaras
- Reinaldo A. C. BianchiRBReinaldo A. C. Bianchi
- 1 October 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7-13
Abstract
In this paper we propose to combine Case-based Reasoning and Heuristically Accelerated Reinforcement Learning to speed up a Reinforcement Learning algorithm in a Transfer Learning problem. To do so, we propose a new algorithm called SARSA Accelerated by Transfer Learning - SATL, which uses Reinforcement Learning to learn how to perform one task, stores the policy for this problem as a case-base and then uses the learned case-base as heuristics to speed up the learning performance in a related, but different, task. A set of empirical evaluations were conducted in transferring the learning between two domains with multiple agents: an expanded version of Littman's simulated robot soccer and the RoboCup Soccer Keep away. A policy learned by one agent in the Littman's soccer is used to speed up the agent learning in the Keep away soccer. The results show that the use of this new algorithm can lead to a significant improvement in the performance of the learning agents.Keywords
This publication has 15 references indexed in Scilit:
- A case-based approach for coordinated action selection in robot soccerArtificial Intelligence, 2009
- Improving Reinforcement Learning by Using Case Based HeuristicsLecture Notes in Computer Science, 2009
- Reinforcement Learning for RoboCup Soccer KeepawayAdaptive Behavior, 2005
- Retrieval, reuse, revision and retention in case-based reasoningThe Knowledge Engineering Review, 2005
- Using Advice to Transfer Knowledge Acquired in One Reinforcement Learning Task to AnotherLecture Notes in Computer Science, 2005
- Markov games as a framework for multi-agent reinforcement learning1994
- Prioritized sweeping: Reinforcement learning with less data and less timeMachine Learning, 1993
- Efficient Learning and Planning Within the Dyna FrameworkAdaptive Behavior, 1993
- Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming1990
- The influence of improvement in one mental function upon the efficiency of other functions: III. Functions involving attention, observation and discrimination.Psychological Review, 1901