Learning to Manipulate Deformable Objects without Demonstrations
Preprint
Abstract
In this paper we tackle the problem of deformable object manipulation through model-free visual reinforcement learning (RL). In order to circumvent the sample inefficiency of RL, we propose two key ideas that accelerate learning. First, we propose an iterative pick-place action space that encodes the conditional relationship between picking and placing on deformable objects. The explicit structural encoding enables faster learning under complex object dynamics. Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points. Then, by selecting the pick point that has Maximal Value under Placing (MVP), we obtain our picking policy. This provides us with an informed picking policy during testing, while using only random pick points during training. Experimentally, this learning framework obtains an order of magnitude faster learning compared to independent action-spaces on our suite of deformable object manipulation tasks with visual RGB observations. Finally, using domain randomization, we transfer our policies to a real PR2 robot for challenging cloth and rope coverage tasks, and demonstrate significant improvements over standard RL techniques on average coverage.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making2022
- Estimating Model Utility for Deformable Object Manipulation Using Multiarmed Bandit MethodsIEEE Transactions on Automation Science and Engineering, 2018
- Three-Dimensional Deformable Object Manipulation Using Fast Online Gaussian Process RegressionIEEE Robotics and Automation Letters, 2018
- Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards2016
- Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours2016
- Team IHMC's Lessons Learned from the DARPA Robotics Challenge TrialsJournal of Field Robotics, 2015
- MoveIt! [ROS Topics]IEEE Robotics & Automation Magazine, 2012
- Tactile sensing for dexterous in-hand manipulation in robotics—A reviewSensors and Actuators A: Physical, 2011
- Tight open knotsThe European Physical Journal E, 2001
- Task-level planning of pick-and-place robot motionsComputer, 1989