Combining self-supervised learning and imitation for vision-based rope manipulation
Authors:
Top Cited Papers
- 1 May 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2146-2153
Abstract
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics. We present a learning-based system where a robot takes as input a sequence of images of a human manipulating a rope from an initial to goal configuration, and outputs a sequence of actions that can reproduce the human demonstration, using only monocular images as input. To perform this task, the robot learns a pixel-level inverse dynamics model of rope manipulation directly from images in a self-supervised manner, using about 60K interactions with the rope collected autonomously by the robot. The human demonstration provides a high-level plan of what to do and the low-level inverse model is used to execute the plan. We show that by combining the high and low-level plans, the robot can successfully manipulate a rope into a variety of target shapes using only a sequence of human-provided images for direction.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- DeepMPC: Learning Deep Latent Features for Model Predictive Control2015
- Human-level control through deep reinforcement learningNature, 2015
- A syntactic approach to robot imitation learning using probabilistic activity grammarsRobotics and Autonomous Systems, 2013
- Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding2010
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural NetworksAdvanced Robotics, 2008
- Knotting/Unknotting Manipulation of Deformable Linear ObjectsThe International Journal of Robotics Research, 2006
- Knot planning from observation2004
- A new point matching algorithm for non-rigid registrationComputer Vision and Image Understanding, 2003
- Learning by watching: extracting reusable task knowledge from visual observation of human performanceIEEE Transactions on Robotics and Automation, 1994