Real-Time Reconstruction of 3-D Tactile Motion Field via Multitask Learning
- 8 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-13
Abstract
The 3-D motion field on the surface of the vision-based tactile sensor contains rich tactile information and serves as the foundation for many downstream tasks. However, achieving real-time and precise reconstruction of 3-D motion field is challenging. In this study, the 3-D motion field is decomposed into depth and 2-D motion field, and a multitask learning model is employed to simultaneously estimate both. The approach excels in accuracy and real-time performance (9.20 ms/frame). For depth reconstruction, the model achieves an average mean absolute error (MAE) of 0.062 mm. For 2-D motion field tracking, an effective structured marker tracking algorithm (SMTA) is introduced, and a marker tracking network (MaTnet) is constructed based on it. This network features dynamic sensing fields and a distance field (DF) auxiliary task, offering strong generalization, interpretability, and ease of training. It exhibits excellent tracking performance in various deformations, with an average error of 1.044 pixels (about 0.03 mm). Finally, the strong transferability of the multitask model is demonstrated, with a maximum decrease in depth reconstruction accuracy of 0.018 mm.Keywords
Funding Information
- National Natural Science Foundation of China (12272220, 52305030)
- Oceanic Interdisciplinary Program of Shanghai Jiao Tong University (SL2021MS017)
- Contemporary Amperex Technology Co., Limited (CATL) Shanghai Research Institute
This publication has 42 references indexed in Scilit:
- GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger2022
- DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction2022
- Tactile and Vision Perception for Intelligent HumanoidsAdvanced Intelligent Systems, 2022
- A Review of Yolo Algorithm DevelopmentsProcedia Computer Science, 2022
- Deep learning for monocular depth estimation: A reviewNeurocomputing, 2021
- Maintaining Grasps within Slipping Bounds by Monitoring Incipient Slip2019
- Dense Tactile Force Estimation using GelSlim and inverse FEM2019
- The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic MorphologiesSoft Robotics, 2018
- Focal Loss for Dense Object Detection2017
- Retrographic sensing for the measurement of surface texture and shape2009