Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains
Authors:
- 22 November 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1048-1055
Abstract
This paper addresses the challenge of terrain-adaptive dynamic locomotion in humanoid robots, traditionally tackled by optimization-based methods or reinforcement learning (RL). Optimization-based methods, such as model-predictive control, excel in finding optimal reaction forces and achieving agile locomotion, but struggle with the nonlinear hybrid dynamics of legged systems and the real-time computation of step location, timing, and reaction forces. Conversely, RL-based methods show promise in navigating dynamic and rough terrains but are limited by their extensive data requirements. We introduce a novel locomotion architecture that integrates a neural network policy, trained through RL in simplified environments, with a state-of-the-art motion controller combining model-predictive control (MPC) and whole-body impulse control (WBIC). The policy efficiently learns high-level locomotion strategies, such as gait selection and step positioning, without the need for full dynamics simulations. This control architecture enables humanoid robots to dynamically navigate discrete terrains, making strategic locomotion decisions (e.g., walking, jumping, and leaping) based on ground height maps. Our results demonstrate that this integrated control architecture achieves dynamic locomotion with significantly fewer training samples than conventional RL-based methods and can be transferred to different humanoid platforms without additional training. The control architecture has been extensively tested in dynamic simulations, accomplishing terrain height-based dynamic locomotion for three different robots.Keywords
All Related Versions
Funding Information
- National Science Foundation
This publication has 32 references indexed in Scilit:
- Learning Torque Control for Quadrupedal Locomotion2023
- Online Optimal Landing Control of the MIT Mini Cheetah2022
- The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors2021
- MPC-based Locomotion Control of Bipedal Robots with Line-Feet Contact using Centroidal Dynamics2021
- Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots2021
- Whole Body Model Predictive Control with a Memory of Motion: Experiments on a Torque-Controlled Talos2021
- MPC-based Controller with Terrain Insight for Dynamic Legged Locomotion2020
- CasADi: a software framework for nonlinear optimization and optimal controlMathematical Programming Computation, 2018
- Functional Neuroanatomy of the Basal GangliaCold Spring Harbor Perspectives in Medicine, 2012
- Trajectory optimization of mechanical hybrid systems using SUMT2006