Real-world humanoid locomotion with reinforcement learning
Top Cited Papers
- 17 April 2024
- journal article
- research article
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 9 (89) , eadi9579
Abstract
Humanoid robots that can autonomously operate in diverse environments have the potential to help address labor shortages in factories, assist elderly at home, and colonize new planets. Although classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesized that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in context, without updating its weights. We trained our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed it to the real-world zero-shot. Our controller could walk over various outdoor terrains, was robust to external disturbances, and could adapt in context.Keywords
All Related Versions
This publication has 45 references indexed in Scilit:
- Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal LocomotionIEEE Access, 2022
- End-to-End Object Detection with Transformers2020
- TALOS: A new humanoid research platform targeted for industrial applications2017
- From 2D Design of Underactuated Bipedal Gaits to 3D Implementation: Walking With Speed TrackingIEEE Access, 2016
- Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robotAutonomous Robots, 2015
- Dynamic arm swinging in human walkingProceedings Of The Royal Society B-Biological Sciences, 2009
- Effects of aging and arm swing on the metabolic cost of stability in human walkingJournal of Biomechanics, 2008
- Effects of suppressing arm swing on kinematics, kinetics, and energetics of human walkingJournal of Biomechanics, 2008
- Biped dynamic walking using reinforcement learningRobotics and Autonomous Systems, 1997
- Patterns of Sagittal Rotation of the Upper Limbs in WalkingPhysical Therapy, 1967