Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics
Authors:
- 30 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11515-11522
Abstract
Robotic fabric manipulation is challenging due to the infinite dimensional configuration space, self-occlusion, and complex dynamics of fabrics. There has been significant prior work on learning policies for specific fabric manipulation tasks, but comparatively less focus on algorithms which can perform many different tasks. We take a step towards this goal by learning point-pair correspondences across different fabric configurations in simulation. Then, given a single demonstration of a new task from an initial fabric configuration, these correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. This makes it possible to define policies to robustly imitate a broad set of multi-step fabric smoothing and folding tasks. The resulting policies achieve 80.3% average task success rate across 10 fabric manipulation tasks on two different physical robotic systems. Results also suggest robustness to fabrics of various colors, sizes, and shapes. See https://tinyurl.com/fabric-descriptors for supplementary material and videos.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Self-Supervised Visual Descriptor Learning for Dense CorrespondenceIEEE Robotics and Automation Letters, 2016
- Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours2016
- Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening2015
- Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning2014
- An open-source research kit for the da Vinci® Surgical System2014
- A geometric approach to robotic laundry foldingThe International Journal of Robotics Research, 2011
- Combining imitation and reinforcement learning to fold deformable planar objects2011
- Model for unfolding laundry using interactive perception2011
- Unfolding of Massive Laundry and Classification Types by Dual ManipulatorJournal of Advanced Computational Intelligence and Intelligent Informatics, 2007
- Collision and self-collision handling in cloth model dedicated to design garments1997