Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization
Open Access
- 23 October 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5556-5563
Abstract
Developing robust vision-guided controllers for quadrupedal robots in complex environments with various obstacles, dynamical surroundings and uneven terrains is very challenging. While Reinforcement Learning (RL) provides a promising paradigm for agile locomotion skills with vision inputs in simulation, it is still very challenging to deploy the vision-guided RL policy in the real world. Our key insight is that the asynchronous multi-modal observations, caused by different latencies in different components of the real robot, create a large sim2real gap for a RL policy. In this paper, we propose Multi-Modal Delay Randomization (MMDR) to address this issue when training in simulation. Specifically, we randomize the selections for both the proprioceptive states and the visual observations in time during training, aiming to simulate the asynchronous inputs when deploying to the real robot. With this technique, we are able to train a RL policy for end-to-end locomotion control in simulation, which can be directly deployed on the real A1 quadruped robot running in the wild. We evaluate our method in different outdoor environments with complex terrain and obstacles. We show that the robot can smoothly maneuver at a high speed while avoiding the obstacles, achieving significant improvement over the baselines. Our project page with videos is at https://mehooz.github.io/mmdr-wild/.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots2018
- Domain randomization for transferring deep neural networks from simulation to the real world2017
- DeepLocoACM Transactions on Graphics, 2017
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids2015
- Bullet physics simulation2015
- MuJoCo: A physics engine for model-based control2012
- Design and use paradigms for gazebo, an open-source multi-robot simulator2005
- Markov decision processes with delays and asynchronous cost collectionIEEE Transactions on Automatic Control, 2003
- Reinforcement Learning in Continuous Time and SpaceNeural Computation, 2000