A Hierarchical Framework for Quadruped Omnidirectional Locomotion Based on Reinforcement Learning
- 18 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (4) , 5367-5378
Abstract
Quadruped locomotion is challenging for many learning-based algorithms. This is because it requires tedious manual tuning to cope with different types of terrains and is difficult to deploy in reality due to the sim-to-real gap between the training and the testing scenarios. This paper proposes a quadruped robot learning system for agile locomotion which does not require any pre-training and works well in various terrains. We introduce a hierarchical framework that uses reinforcement learning as the high-level policy to adjust the low-level trajectory generator for a better adaptability to various terrains. We compact the observation and the action spaces of reinforcement learning to deploy the proposed framework on a host computer interfaced with the robot. Besides, we design an omnidirectional trajectory generator guided by robot posture, which generates omnidirectional foot trajectories to interact with the environment. Experimental results and the supplementary video demonstrate that our hierarchical framework only trained in simulation can be easily deployed in the real world, and also has the advantages of fast convergence and good terrain adaptability. Note to Practitioners —This paper presents a hierarchical framework for quadruped robots. It combines a high-level reinforcement learning controller with a posture-guided trajectory generator to adaptively generate omnidirectional motions. Our method is easy to train as it converges fast and does not need to adjust a dozen or so of rewards. The quadruped robot can be deployed in a real environment directly after being trained in simulation. With the trained hierarchical framework deployed on a remote host computer, the robot works well in a variety of real-world environments unseen in the simulation.Keywords
Funding Information
- National Key Research and Development Plan of China (2021ZD0112002, 2018AAA0102504)
- National Natural Science Foundation of China (U1913204, U22A2057)
- Natural Science Foundation of Shandong Province for Distinguished Young Scholars (ZR2020JQ29)
- Project for Self-Developed Innovation Team of Jinan City (2021GXRC038)
This publication has 46 references indexed in Scilit:
- STANCE: Locomotion Adaptation Over Soft TerrainIEEE Transactions on Robotics, 2020
- Dynamic Locomotion on Slippery GroundIEEE Robotics and Automation Letters, 2019
- Distributed Learning of Decentralized Control Policies for Articulated Mobile RobotsIEEE Transactions on Robotics, 2019
- Mini Cheetah: A Platform for Pushing the Limits of Dynamic Quadruped Control2019
- Workspace CPG with Body Pose Control for Stable, Directed Vision during Omnidirectional Locomotion2019
- Contact Force/Torque Control Based on Viscoelastic Model for Stable Bipedal Walking on Indefinite Uneven TerrainIEEE Transactions on Automation Science and Engineering, 2019
- Learning agile and dynamic motor skills for legged robotsScience Robotics, 2019
- Coarse-to-Fine UAV Target Tracking With Deep Reinforcement LearningIEEE Transactions on Automation Science and Engineering, 2018
- ANYmal - toward legged robots for harsh environmentsAdvanced Robotics, 2017
- Cerebellar Contributions to Locomotor Adaptations during Splitbelt Treadmill WalkingThe Journal of Neuroscience, 2006