Visual Reinforcement Learning With Self-Supervised 3D Representations
- 20 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (5) , 2890-2897
Abstract
A prominent approach to visual Reinforcement Learning (RL) is to learn an internal state representation using self-supervised methods, which has the potential benefit of improved sample-efficiency and generalization through additional learning signal and inductive biases. However, while the real world is inherently 3D, prior efforts have largely been focused on leveraging 2D computer vision techniques as auxiliary self-supervision. In this work, we present a unified framework for self-supervised learning of 3D representations for motor control. Our proposed framework consists of two phases: a pretraining phase where a deep voxel-based 3D autoencoder is pretrained on a large object-centric dataset, and a finetuning phase where the representation is jointly finetuned together with RL on in-domain data. We empirically show that our method enjoys improved sample efficiency compared to 2D representation learning methods. Additionally, our learned policies transfer zero-shot to a real robot setup with only approximate geometric correspondence, and successfully solve motor control tasks that involve grasping and lifting from a single, uncalibrated RGB camera .Keywords
All Related Versions
Funding Information
- Amazon Research Award and gifts from Qualcomm
This publication has 16 references indexed in Scilit:
- RMA: Rapid Motor Adaptation for Legged Robots2021
- Auto-Tuned Sim-to-Real Transfer2021
- DROID: Minimizing the Reality Gap Using Single-Shot Human DemonstrationIEEE Robotics and Automation Letters, 2021
- 6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints2020
- Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras2020
- Learning dexterous in-hand manipulationThe International Journal of Robotics Research, 2019
- Multi-View PointNet for 3D Scene Understanding2019
- Learning Spatial Common Sense With Geometry-Aware Recurrent Networks2019
- Distance Modulation of Neural Activity in the Visual CortexScience, 1998
- Least-squares estimation of transformation parameters between two point patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1991