Self-organizing maps for storage and transfer of knowledge in reinforcement learning
- 20 December 2018
- journal article
- research article
- Published by SAGE Publications in Adaptive Behavior
- Vol. 27 (2) , 111-126
Abstract
The idea of reusing or transferring information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency of a reinforcement learning agent. In this work, we describe a novel approach for reusing previously acquired knowledge by using it to guide the exploration of an agent while it learns new tasks. In order to do so, we employ a variant of the growing self-organizing map algorithm, which is trained using a measure of similarity that is defined directly in the space of the vectorized representations of the value functions. In addition to enabling transfer across tasks, the resulting map is simultaneously used to enable the efficient storage of previously acquired task knowledge in an adaptive and scalable manner. We empirically validate our approach in a simulated navigation environment and also demonstrate its utility through simple experiments using a mobile micro-robotics platform. In addition, we demonstrate the scalability of this approach and analytically examine its relation to the proposed network growth mechanism. Furthermore, we briefly discuss some of the possible improvements and extensions to this approach, as well as its relevance to real-world scenarios in the context of continual learning.Keywords
All Related Versions
Funding Information
- Ministry of Education, Singapore (President’s Graduate Fellowship)
This publication has 13 references indexed in Scilit:
- Identification and off-policy learning of multiple objectives using adaptive clusteringNeurocomputing, 2017
- Self-Organizing Neural Networks Integrating Domain Knowledge and Reinforcement LearningIEEE Transactions on Neural Networks and Learning Systems, 2014
- Learning domain structure through probabilistic policy reuse in reinforcement learningProgress in Artificial Intelligence, 2012
- Continuous state/action reinforcement learning: A growing self-organizing map approachNeurocomputing, 2011
- Task similarity measures for transfer in reinforcement learning task libraries2006
- A teaching method using a self-organizing map for reinforcement learningArtificial Life and Robotics, 2004
- Applications of the self-organising map to reinforcement learningNeural Networks, 2002
- Dynamic self-organizing maps with controlled growth for knowledge discoveryIEEE Transactions on Neural Networks, 2000
- The self-organizing mapNeurocomputing, 1998
- Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge1998