Accelerating Deep Continuous Reinforcement Learning through Task Simplification
Authors:
- 1 July 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Robotic motor policies can, in theory, be learned via deep continuous reinforcement learning. In practice, however, collecting the enormous amount of required training samples in realistic time, surpasses the possibilities of many robotic platforms. To address this problem, we propose a novel method for accelerating the learning process by task simplification inspired by the Goldilocks effect known from developmental psychology. We present results on a reach-for-grasp task that is learned with the Deep Deterministic Policy Gradients (DDPG) algorithm. Task simplification is realized by initially training the system with “larger-than-life” training objects that adapt their reachability dynamically during training. We achieve a significant acceleration compared to the unaltered training setup. We describe modifications to the DDPG algorithm with regard to the replay buffer to prevent artifacts during the learning process from the simplified learning instances while maintaining the speed of learning. With this result, we contribute towards the realistic application of deep reinforcement learning on robotic platforms.Keywords
This publication has 13 references indexed in Scilit:
- Empagliflozin for Type 2 Diabetes Mellitus: An Overview of Phase 3 Clinical TrialsCurrent Diabetes Reviews, 2017
- A Modular Software Framework for Eye–Hand Coordination in Humanoid RobotsFrontiers in Robotics and AI, 2016
- Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours2016
- Human-level control through deep reinforcement learningNature, 2015
- The Goldilocks Effect in Infant Auditory AttentionChild Development, 2014
- Size matters: How age and reaching experiences shape infants’ preferences for different sized objectsInfant Behavior and Development, 2013
- Reinforcement Learning in Continuous Action Spaces2007
- The development of reaching and looking preferences in infants to objects of different sizes.Developmental Psychology, 2001
- Prioritized sweeping: Reinforcement learning with less data and less timeMachine Learning, 1993
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989