High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures
Authors:
- 19 May 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9981-9988
Abstract
This work presents an overview of the techni-cal details behind a high-performance reinforcement learning policy deployment with the Spot RL Researcher Development Kit for low-level motor access on Boston Dynamic's Spot. This represents the first public demonstration of an end-to-end reinforcement learning policy deployed on Spot hardware with training code publicly available through Nvidia IsaacLab and deployment code available through Boston Dynamics. We utilize Wasserstein Distance and Maximum Mean Discrepancy to quantify the distributional dissimilarity of data collected on hardware and in simulation to measure our sim-to-real gap. We use these measures as a scoring function for the Covariance Matrix Adaptation Evolution Strategy to optimize simulated parameters that are unknown or difficult to measure from Spot. Our procedure for modeling and training produces high-quality reinforcement learning policies capable of multiple gaits, including a flight phase. We deploy policies capable of over 5.2m/s locomotion, more than triple Spot's default controller maximum speed, robustness to slippery surfaces, disturbance rejection, and overall agility previously unseen on Spot. We detail our method and release our code to support future work on Spot with the low-level API.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots2021
- Policy Transfer via Kinematic Domain Randomization and Adaptation2021
- Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks2019
- Mini Cheetah: A Platform for Pushing the Limits of Dynamic Quadruped Control2019
- Pros and cons of GAN evaluation measuresComputer Vision and Image Understanding, 2019
- Real-time motion planning of legged robots: A model predictive control approach2017
- Multiple task optimization with a mixture of controllers for motion generation2015
- Kicking motion planning of Nao robots based on CMA-ES2015
- Estimation of inertial parameters of rigid body links of manipulators1985
- The Wasserstein distance and approximation theoremsProbability Theory and Related Fields, 1985