RL-Augmented MPC for Non-Gaited Legged and Hybrid Locomotion
- 19 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (5) , 5797-5804
Abstract
We propose a contact-explicit hierarchical architecture coupling Reinforcement Learning (RL) and Model Predictive Control (MPC), where a high-level RL agent provides gait and navigation commands to a low-level locomotion MPC. This offloads the combinatorial burden of contact timing from the MPC by learning acyclic gaits through trial and error in simulation. We show that only a minimal set of rewards and limited tuning are required to obtain effective policies. We validate the architecture in simulation across robotic platforms spanning $\mathbf {50}\,$kg to $\mathbf {120}\,$kg and different MPC implementations, observing the emergence of acyclic gaits and timing adaptations in flat-terrain legged and hybrid locomotion, and further demonstrating extensibility to non-flat terrains. Across all platforms, we achieve zero-shot sim-to-sim transfer without domain randomization, and we further demonstrate zero-shot sim-to-real transfer without domain randomization on Centauro, our $\mathbf {120}\,$kg wheeled-legged humanoid robot.Keywords
All Related Versions
Funding Information
- Horizon Europe Framework Programme project euROBIN
This publication has 20 references indexed in Scilit:
- Non-Gaited Legged Locomotion With Monte-Carlo Tree Search and Supervised LearningIEEE Robotics and Automation Letters, 2024
- CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal ControlIEEE Robotics and Automation Letters, 2024
- RL + Model-Based Control: Using On-Demand Optimal Control to Learn Versatile Legged LocomotionIEEE Robotics and Automation Letters, 2023
- The XBot2 real-time middleware for roboticsRobotics and Autonomous Systems, 2023
- RLOC: Terrain-Aware Legged Locomotion Using Reinforcement Learning and Optimal ControlIEEE Transactions on Robotics, 2022
- Whole-Body MPC and Online Gait Sequence Generation for Wheeled-Legged Robots2021
- Rolling in the Deep – Hybrid Locomotion for Wheeled-Legged Robots Using Online Trajectory OptimizationIEEE Robotics and Automation Letters, 2020
- Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control2018
- Whole-Body Nonlinear Model Predictive Control Through Contacts for QuadrupedsIEEE Robotics and Automation Letters, 2018
- Synthesis and stabilization of complex behaviors through online trajectory optimization2012