KETO: Learning Keypoint Representations for Tool Manipulation
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7278-7285
Abstract
We aim to develop an algorithm for robots to manipulate novel objects as tools for completing different task goals. An efficient and informative representation would facilitate the effectiveness and generalization of such algorithms. For this purpose, we present KETO, a framework of learning keypoint representations of tool-based manipulation. For each task, a set of task-specific keypoints is jointly predicted from 3D point clouds of the tool object by a deep neural network. These keypoints offer a concise and informative description of the object to determine grasps and subsequent manipulation actions. The model is learned from self-supervised robot interactions in the task environment without the need for explicit human annotations. We evaluate our framework in three manipulation tasks with tool use. Our model consistently outperforms state-of-the-art methods in terms of task success rates. Qualitative results of keypoint prediction and tool generation are shown to visualize the learned representations.Keywords
All Related Versions
This publication has 45 references indexed in Scilit:
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Learning human activities and object affordances from RGB-D videosThe International Journal of Robotics Research, 2013
- 3D facial expression recognition using SIFT descriptors of automatically detected keypointsThe Visual Computer, 2011
- Gravity-Based Robotic Cloth Folding2010
- Grasping the affordances, understanding the reasoning: Toward a dialectical theory of human tool use.Psychological Review, 2010
- To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot ControlAdaptive Behavior, 2007
- Keypoint Detection and Local Feature Matching for Textured 3D Face RecognitionInternational Journal of Computer Vision, 2007
- Learning Features for Tracking2007
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Tools and Human EvolutionScientific American, 1960