Learning Humanoid Locomotion with Transformers
Authors:
Preprint
Abstract
We present a sim-to-real learning-based approach for real-world humanoid locomotion. Our controller is a causal Transformer trained by autoregressive prediction of future actions from the history of observations and actions. We hypothesize that the observation-action history contains useful information about the world that a powerful Transformer model can use to adapt its behavior in-context, without updating its weights. We do not use state estimation, dynamics models, trajectory optimization, reference trajectories, or pre-computed gait libraries. Our controller is trained with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed to the real world in a zero-shot fashion. We evaluate our approach in high-fidelity simulation and successfully deploy it to the real robot as well. To the best of our knowledge, this is the first demonstration of a fully learning-based method for real-world full-sized humanoid locomotion.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Reinforcement Learning-Based Cascade Motion Policy Design for Robust 3D Bipedal LocomotionIEEE Access, 2022
- The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors2021
- Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition2021
- End-to-End Object Detection with Transformers2020
- Sim-to-Real Transfer of Robotic Control with Dynamics Randomization2018
- Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition2018
- TALOS: A new humanoid research platform targeted for industrial applications2017
- Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robotAutonomous Robots, 2015
- Synthesis of natural arm swing motion in human bipedal walkingJournal of Biomechanics, 2008
- Biped dynamic walking using reinforcement learningRobotics and Autonomous Systems, 1997