Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtasks
Authors:
- Luca MarzariLMLuca Marzari
- Ameya PoreAPAmeya Pore
- Diego Dall'AlbaDDDiego Dall'Alba
- Gerardo Aragon-CamarasaGAGerardo Aragon-Camarasa
- Alessandro FarinelliAFAlessandro Farinelli
- Paolo FioriniPFPaolo Fiorini
Preprint
Abstract
Deep Reinforcement Learning (DRL) is emerging as a promising approach to generate adaptive behaviors for robotic platforms. However, a major drawback of using DRL is the data-hungry training regime that requires millions of trial and error attempts, which is impractical when running experiments on robotic systems. To address this issue, we propose a multi-subtask reinforcement learning method where complex tasks are decomposed manually into low-level subtasks by leveraging human domain knowledge. These subtasks can be parametrized as expert networks and learned via existing DRL methods. Trained subtasks can then be composed with a high-level choreographer. As a testbed, we use a pick and place robotic simulator to demonstrate our methodology, and show that our method outperforms an imitation learning-based method and reaches a high success rate compared to an end-to-end learning approach. Moreover, we transfer the learned behavior in a different robotic environment that allows us to exploit sim-to-real transfer and demonstrate the trajectories in a real robotic system. Our training regime is carried out using a central processing unit (CPU)-based system, which demonstrates the data-efficient properties of our approach.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Multi-expert learning of adaptive legged locomotionScience Robotics, 2020
- Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments2020
- Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation2019
- Learning to Walk Via Deep Reinforcement Learning2019
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- Recent Advances in Hierarchical Reinforcement LearningDiscrete Event Dynamic Systems, 2003
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991