Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning
Authors:
- 20 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7-12
Abstract
Current research directions in deep reinforcement learning include bridging the simulation-reality gap, improving sample efficiency of experiences in distributed multi-agent reinforcement learning, together with the development of robust methods against adversarial agents in distributed learning, among many others. In this work, we are particularly interested in analyzing how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems where the operation of the different robots is not necessarily homogeneous. These variations can happen due to sensing mismatches, inherent errors in terms of calibration of the mechanical joints, or simple differences in accuracy. While our results are simulation-based, we introduce the effect of sensing, calibration, and accuracy mismatches in distributed reinforcement learning with proximal policy optimization (PPO). We discuss on how both the different types of perturbances and how the number of agents experiencing those perturbances affect the collaborative learning effort. The simulations are carried out using a Kuka arm model in the Bullet physics engine. This is, to the best of our knowledge, the first work exploring the limitations of PPO in multi-robot systems when considering that different robots might be exposed to different environments where their sensors or actuators have induced errors. With the conclusions of this work, we set the initial point for future work on designing and developing methods to achieve robust reinforcement learning on the presence of real-world perturbances that might differ within a multi-robot system.Keywords
All Related Versions
Funding Information
- Research Council of Finland
This publication has 10 references indexed in Scilit:
- Reinforcement Learning with Perturbed RewardsProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Enhancing Autonomy with Blockchain and Multi-Access Edge Computing in Distributed Robotic Systems2020
- Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and ApplicationsIEEE Transactions on Cybernetics, 2020
- Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning2018
- Cooperative Multi-agent Control Using Deep Reinforcement Learning2017
- Deep Reinforcement Learning: A Brief SurveyIEEE Signal Processing Magazine, 2017
- Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks2017
- An application of reinforcement learning algorithms to industrial multi-robot stations for cooperative handling operation2017
- Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates2017
- Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning2017