Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtasks
Authors:
- 6 December 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 640-645
Abstract
Deep Reinforcement Learning (DRL) is emerging as a promising approach to generate adaptive behaviors for robotic platforms. However, a major drawback of using DRL is the data-hungry training regime that requires millions of trial and error attempts, which is impractical when running experiments on robotic systems. Learning from Demonstrations (LfD) has been introduced to solve this issue by cloning the behavior of expert demonstrations. However, LfD requires a large number of demonstrations that are difficult to be acquired since dedicated complex setups are required. To overcome these limitations, we propose a multi-subtask reinforcement learning methodology where complex pick and place tasks can be decomposed into low-level subtasks. These subtasks are parametrized as expert networks and learned via DRL methods. Trained subtasks are then combined by a high-level choreographer to accomplish the intended pick and place task considering different initial configurations. As a testbed, we use a pick and place robotic simulator to demonstrate our methodology and show that our method outperforms a benchmark methodology based on LfD in terms of sample-efficiency. We transfer the learned policy to the real robotic system and demonstrate robust grasping using various geometric-shaped objects.Keywords
All Related Versions
Funding Information
- ARS
This publication has 12 references indexed in Scilit:
- On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning2020
- Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation2019
- Toward Modularization of Neural Network Autonomous Driving Policy Using Parallel Attribute Networks2019
- Overcoming Exploration in Reinforcement Learning with Demonstrations2018
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates2017
- Learning modular neural network policies for multi-task and multi-robot transfer2017
- Recent Advances in Hierarchical Reinforcement LearningDiscrete Event Dynamic Systems, 2003
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991
- Intelligence without representationArtificial Intelligence, 1991