Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks
Preprint
Abstract
Randomization is currently a widely used approach in Sim2Real transfer for data-driven learning algorithms in robotics. Still, most Sim2Real studies report results for a specific randomization technique and often on a highly customized robotic system, making it difficult to evaluate different randomization approaches systematically. To address this problem, we define an easy-to-reproduce experimental setup for a robotic reach-and-balance manipulator task, which can serve as a benchmark for comparison. We compare four randomization strategies with three randomized parameters both in simulation and on a real robot. Our results show that more randomization helps in Sim2Real transfer, yet it can also harm the ability of the algorithm to find a good policy in simulation. Fully randomized simulations and fine-tuning show differentiated results and translate better to the real robot than the other approaches tested.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Robot Learning From Randomized Simulations: A ReviewFrontiers in Robotics and AI, 2022
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- Improving Robot Motor Learning with Negatively Valenced Reinforcement SignalsFrontiers in Neurorobotics, 2017
- Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility StudyIEEE Transactions on Medical Imaging, 2016
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- The Transferability Approach: Crossing the Reality Gap in Evolutionary RoboticsIEEE Transactions on Evolutionary Computation, 2013
- Back to reality: Crossing the reality gap in evolutionary roboticsIFAC Proceedings Volumes, 2004
- Interaction in reinforcement learning reduces the need for finely tuned hyperparameters in complex tasks