Bi-Directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents
- 25 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 2634-2641
Abstract
Deep reinforcement learning models are notoriously data hungry, yet real-world data is expensive and time consuming to obtain. The solution that many have turned to is to use simulation for training before deploying the robot in a real environment. Simulation offers the ability to train large numbers of robots in parallel, and offers an abundance of data. However, no simulation is perfect, and robots trained solely in simulation fail to generalize to the real-world, resulting in a “sim-vs-real gap”. How can we overcome the trade-off between the abundance of less accurate, artificial data from simulators and the scarcity of reliable, real-world data? In this letter, we propose Bi-directional Domain Adaptation (BDA), a novel approach to bridge the sim-vs-real gap in both directions- real2sim to bridge the visual domain gap, and sim2real to bridge the dynamics domain gap. We demonstrate the benefits of BDA on the task of PointGoal Navigation. BDA with only 5 k real-world (state, action, next-state) samples matches the performance of a policy fine-tuned with ~ 600 k samples, resulting in a speed-up of ~ 120×.Keywords
All Related Versions
Funding Information
- National Science Foundation
- AFRL
- DARPA
- ONR YIPs
- ARO PECASE
- National Science Foundation (DGE-1650044)
- Google Women Techmaker's Fellowship
This publication has 15 references indexed in Scilit:
- Residual Reinforcement Learning for Robot Control2019
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience2019
- VR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual ControlIEEE Robotics and Automation Letters, 2019
- Gibson Env: Real-World Perception for Embodied Agents2018
- Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping2018
- Sim-to-Real Transfer of Robotic Control with Dynamics Randomization2018
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Domain randomization for transferring deep neural networks from simulation to the real world2017
- CAD2RL: Real Single-Image Flight Without a Single Real Image2017
- A flexible and scalable SLAM system with full 3D motion estimation2011