Criminisi Algorithm Applied to a GelSight Fingertip Sensor for Multi-modality Perception
- 1 July 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 190-195
Abstract
Tactile sensing is an important perception mode for a robot to interact with the external environment and perform tasks such as object manipulation. However, there are still limitations for current artificial tactile sensing technologies to perform manipulation tasks as well as a human. In this paper, we propose a biomimetic tactile sensor based on the principle of the GelSight fingertip tactile sensor [1], but following a different architecture design and using a different image processing algorithm. The contribution of our proposed sensor is to concurrently detect contact force and rebuild the object shape. In this paper, we focus on generalizing the tactile-related information from the force-related information by using the Criminisi algorithm [2]. Concerning the hardware, we use a combination of a coated silicone elastomer (with a marker array across its surface) and an internal spring structure to determine force information. A camera placed at the bottom of the sensor housing captures the deformation of the elastomer surface illuminated by an internal light. With regards to the software, we analyze the position and shape changes of the marker array to distinguish different types of contact forces, however, the use of the markers causes interference in the shape-recognition step, so we apply the Criminisi algorithm to remove the force markers from the captured images and restore the object contour image. We use a Convolutional Neural Network (CNN) to determine the shape of the handled object based on the captured deformation; we achieve an accuracy of 89.6% for shape classification.Keywords
This publication has 20 references indexed in Scilit:
- Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials2017
- Criminisi-Based Sparse Representation for Image Inpainting2017
- Piezoresistive Tactile Sensor Discriminating Multidirectional ForcesSensors, 2015
- Measurement of shear and slip with a GelSight tactile sensor2015
- Integration of Force Reflection with Tactile Sensing for Minimally Invasive Robotics-Assisted Tumor LocalizationIEEE Transactions on Haptics, 2012
- 3D flexible tactile sensor using electromagnetic induction coils2012
- Microgeometry capture using an elastomeric sensorACM Transactions on Graphics, 2011
- Tactile Sensing—From Humans to HumanoidsIEEE Transactions on Robotics, 2010
- Normal and Shear Force Measurement Using a Flexible Polymer Tactile Sensor With Embedded Multiple CapacitorsJournal of Microelectromechanical Systems, 2008
- A capacitive tactile sensor for shear and normal force measurementsSensors and Actuators A: Physical, 1992