Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
Preprint
Abstract
Data-efficiency is crucial for autonomous robots to adapt to new tasks and environments. In this work we focus on robotics problems with a budget of only 10-20 trials. This is a very challenging setting even for data-efficient approaches like Bayesian optimization (BO), especially when optimizing higher-dimensional controllers. Simulated trajectories can be used to construct informed kernels for BO. However, previous work employed supervised ways of extracting low-dimensional features for these. We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way. We further compress the search space for BO by reducing exploration in parts of the state space that are undesirable, without requiring explicit constraints on controller parameters. We validate our approach with hardware experiments on a Daisy hexapod robot and an ABB Yumi manipulator. We also present simulation experiments with further comparisons to several baselines on Daisy and two manipulators. Our experiments indicate the proposed trajectory-based kernel with dynamic compression can offer ultra data-efficient optimization.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience2019
- Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped2019
- State representation learning for control: An overviewNeural Networks, 2018
- Automatic Chemical Design Using a Data-Driven Continuous Representation of MoleculesACS Central Science, 2018
- Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE, 2015
- Robots that can adapt like animalsNature, 2015
- Optimization‐based Full Body Control for the DARPA Robotics ChallengeJournal of Field Robotics, 2015
- Active learning of visual descriptors for grasping using non-parametric smoothed beta distributionsRobotics and Autonomous Systems, 2012
- Combining active learning and reactive control for robot graspingRobotics and Autonomous Systems, 2010
- Online Optimization of Swimming and Crawling in an Amphibious Snake RobotIEEE Transactions on Robotics, 2008