Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception
- 9 April 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 32 (3) , 1192-1203
Abstract
The material attribute of an object's surface is critical to enable robots to perform dexterous manipulations or actively interact with their surrounding objects. Tactile sensing has shown great advantages in capturing material properties of an object's surface. However, the conventional classification method based on tactile information may not be suitable to estimate or infer material properties, particularly during interacting with unfamiliar objects in unstructured environments. Moreover, it is difficult to intuitively obtain material properties from tactile data as the tactile signals about material properties are typically dynamic time sequences. In this article, a visual-tactile cross-modal learning framework is proposed for robotic material perception. In particular, we address visual-tactile cross-modal learning in the lifelong learning setting, which is beneficial to incrementally improve the ability of robotic cross-modal material perception. To this end, we proposed a novel lifelong cross-modal learning model. Experimental results on the three publicly available data sets demonstrate the effectiveness of the proposed method.Keywords
Funding Information
- National Key Research and Development Program of China (2018YFB1305102)
- Tsinghua University, Beijing, China
This publication has 59 references indexed in Scilit:
- Biologically Inspired Visual Model With Preliminary Cognition and Active Attention AdjustmentIEEE Transactions on Cybernetics, 2015
- Deep correspondence restricted Boltzmann machine for cross-modal retrievalNeurocomputing, 2015
- The Concept of “Attractive Region in Environment” and its Application in High-Precision Tasks With Low-Precision SystemsIEEE/ASME Transactions on Mechatronics, 2014
- Transfer Learning for Visual Categorization: A SurveyIEEE Transactions on Neural Networks and Learning Systems, 2014
- Visual perception of materials and their propertiesVision Research, 2014
- A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their SemanticsInternational Journal of Computer Vision, 2013
- A three-network architecture for on-line learning and optimization based on adaptive dynamic programmingNeurocomputing, 2012
- Dimensional Reduction of High-Frequency Accelerations for Haptic Rendering2010
- Canonical Correlation Analysis: An Overview with Application to Learning MethodsNeural Computation, 2004
- Lifelong robot learningRobotics and Autonomous Systems, 1995