What Can I Do With This Tool? Self-Supervised Learning of Tool Affordances From Their 3-D Geometry
- 19 June 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 10 (3) , 595-610
Abstract
The ability to use tools can significantly increase the range of activities that an agent is capable of. Humans start using external objects since an early age to accomplish their goals, learning from interaction and observation the relationship between the objects used, their own actions, and the resulting effects, i.e., the tool affordances. Robots capable of autonomously learning affordances in a similar self-supervised way would be far more versatile and simpler to design than purpose-specific ones. This paper proposes and evaluates an approach to allow robots to learn tool affordances from interaction, and generalize them among similar tools based on their 3-D geometry. A set of actions is performed by the iCub robot with a large number of tools grasped in different poses, and the effects observed. Tool affordances are learned as a regression between tool-pose features and action-effect vector projections on respective self-organizing maps, which enables the system to avoid categorization and keep gradual representations of both elements. Moreover, we propose a set of robot-centric 3-D tool descriptors, and study their suitability for interaction scenarios, comparing also their performance against features derived from deep convolutional neural networks. Results show that the presented methods allow the robot to predict the effect of its tool use actions accurately, even for previously unseen tool and poses, and thereby to select the best action for a particular goal given a tool-pose.Keywords
Funding Information
- European FP7 ICT (270273, 288382)
This publication has 44 references indexed in Scilit:
- Active learning of inverse models with intrinsically motivated goal exploration in robotsRobotics and Autonomous Systems, 2013
- Object–Action Complexes: Grounded abstractions of sensory–motor processesRobotics and Autonomous Systems, 2011
- Interactive object recognition using proprioceptive and auditory feedbackThe International Journal of Robotics Research, 2011
- Goal emulation and planning in perceptual space using learned affordancesRobotics and Autonomous Systems, 2011
- To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot ControlAdaptive Behavior, 2007
- Parietofrontal Circuits for Action and Space Perception in the Macaque MonkeyNeuroImage, 2001
- PERCEPTUAL LEARNINGAnnual Review of Psychology, 1998
- Affordances and Prospective Control: An Outline of the OntologyEcological Psychology, 1992
- A general regression neural networkIEEE Transactions on Neural Networks, 1991
- The self-organizing mapProceedings of the IEEE, 1990