Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion
- 29 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (11) , 7799-7806
Abstract
Control of legged robots is a challenging problem that has been investigated by different approaches, such as model-based control and learning algorithms. This work proposes a novel Imitating and Finetuning Model Predictive Control (IFM) framework to take the strengths of both approaches. Our framework first develops a conventional model predictive controller (MPC) using Differential Dynamic Programming and Raibert heuristic, which serves as an expert policy. Then we train a clone of the MPC using imitation learning to make the controller learnable. Finally, we leverage deep reinforcement learning with limited exploration for further finetuning the policy on more challenging terrains. By conducting comprehensive simulation and hardware experiments, we demonstrate that the proposed IFM framework can significantly improve the performance of the given MPC controller on rough, slippery, and conveyor terrains that require careful coordination of footsteps. We also showcase that IFM can efficiently produce more symmetric, periodic, and energy-efficient gaits compared to Vanilla RL with a minimal burden of reward shaping.Keywords
All Related Versions
Funding Information
- Korea Evaluation Institute of Industrial Technology (20018216)
- Development of Mobile Intelligence SW
- MIT Biomimetic Robotics Lab
- Naver Labs Corporation
This publication has 32 references indexed in Scilit:
- Learning agile and dynamic motor skills for legged robotsScience Robotics, 2019
- CIRL: Controllable Imitative Reinforcement Learning for Vision-Based Self-driving2018
- DeepMimicACM Transactions on Graphics, 2018
- Agile Autonomous Driving using End-to-End Deep Imitation Learning2018
- Robust Rough-Terrain Locomotion with a Quadrupedal Robot2018
- Per-Contact Iteration Method for Solving Contact DynamicsIEEE Robotics and Automation Letters, 2018
- Control-limited differential dynamic programming2014
- Control of dynamic gaits for a quadrupedal robot2013
- A Review of New Analytic Techniques for Quantifying Symmetry in LocomotionSymmetry, 2010
- Legged Robots That BalanceIEEE Expert, 1986