Deep Kernels for Optimizing Locomotion Controllers
Preprint
Abstract
Sample efficiency is important when optimizing parameters of locomotion controllers, since hardware experiments are time consuming and expensive. Bayesian Optimization, a sample-efficient optimization framework, has recently been widely applied to address this problem, but further improvements in sample efficiency are needed for practical applicability to real-world robots and high-dimensional controllers. To address this, prior work has proposed using domain expertise for constructing custom distance metrics for locomotion. In this work we show how to learn such a distance metric automatically. We use a neural network to learn an informed distance metric from data obtained in high-fidelity simulations. We conduct experiments on two different controllers and robot architectures. First, we demonstrate improvement in sample efficiency when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We then conduct simulation experiments to optimize a 16-dimensional controller for a 7-link robot model and obtain significant improvements even when optimizing in perturbed environments. This demonstrates that our approach is able to enhance sample efficiency for two different controllers, hence is a fitting candidate for further experiments on hardware in the future.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Walking and Running with Passive Compliance: Lessons from Engineering: A Live Demonstration of the ATRIAS BipedIEEE Robotics & Automation Magazine, 2018
- Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization2017
- Experimental Evaluation of Deadbeat Running on the ATRIAS BipedIEEE Robotics and Automation Letters, 2017
- Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE, 2015
- Momentum control with hierarchical inverse dynamics on a torque-controlled humanoidAutonomous Robots, 2015
- Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robotAutonomous Robots, 2015
- Bayesian optimization for learning gaits under uncertaintyAnnals of Mathematics and Artificial Intelligence, 2015
- Robots that can adapt like animalsNature, 2015
- Optimization‐based Full Body Control for the DARPA Robotics ChallengeJournal of Field Robotics, 2015
- Legged Robots That BalanceIEEE Expert, 1986