Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images
- 26 September 2019
- journal article
- research article
- Published by Cambridge University Press (CUP) in Robotica
- Vol. 38 (7) , 1242-1262
Abstract
SUMMARY: The study of dexterous manipulation has provided important insights into human sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here, we describe a method using the deformation and color distribution of the fingernail and its surrounding skin to estimate the fingertip forces, torques, and contact surface curvatures for various objects, including the shape and material of the contact surfaces and the weight of the objects. The proposed method circumvents limitations associated with sensorized objects, gloves, or fixed contact surface type. In addition, compared with previous single finger estimation in an experimental environment, we extend the approach to multiple finger force estimation, which can be used for applications such as human grasping analysis. Four algorithms are used, c.q., Gaussian process, convolutional neural networks, neural networks with fast dropout, and recurrent neural networks with fast dropout, to model a mapping from images to the corresponding labels. The results further show that the proposed method has high accuracy to predict force, torque, and contact surface.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Measuring fingertip forces from camera images for random finger poses2015
- Optimizing Fingernail Imaging Calibration for 3D Force Magnitude PredictionIEEE Transactions on Haptics, 2015
- Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and frictionScientific Data, 2014
- Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian processes2014
- Computing grip force and torque from finger nail images using Gaussian processes2013
- Force prediction by fingernail imaging using active appearance models2013
- Human Ability to Discriminate Direction of Three-Dimensional Force Stimuli Applied to the Finger PadJournal of Neurophysiology, 2011
- Sensorimotor Control of Manipulation2009
- EigenNail for Finger Force Direction RecognitionProceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (cat. No.01ch37164), 2007
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998