Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation
Authors:
Preprint
Abstract
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics. We present a learning-based system where a robot takes as input a sequence of images of a human manipulating a rope from an initial to goal configuration, and outputs a sequence of actions that can reproduce the human demonstration, using only monocular images as input. To perform this task, the robot learns a pixel-level inverse dynamics model of rope manipulation directly from images in a self-supervised manner, using about 60K interactions with the rope collected autonomously by the robot. The human demonstration provides a high-level plan of what to do and the low-level inverse model is used to execute the plan. We show that by combining the high and low-level plans, the robot can successfully manipulate a rope into a variety of target shapes using only a sequence of human-provided images for direction.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- The Curious Robot: Learning Visual Representations via Physical Interactions2016
- Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours2016
- A syntactic approach to robot imitation learning using probabilistic activity grammarsRobotics and Autonomous Systems, 2013
- A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario2013
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural NetworksAdvanced Robotics, 2008
- Knotting/Unknotting Manipulation of Deformable Linear ObjectsThe International Journal of Robotics Research, 2006
- A new algorithm for non-rigid point matching2002
- Learning by watching: extracting reusable task knowledge from visual observation of human performanceIEEE Transactions on Robotics and Automation, 1994
- Introduction to Knot Theory1977