Toward Vision-Based High Sampling Interaction Force Estimation With Master Position and Orientation for Teleoperation
- 7 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 6640-6646
Abstract
In this study, a vision-based high sampling rate interaction force estimation method is proposed for teleoperation systems that uses master position and orientation information without using physical sensors such as force/torque (F/T) or tactile sensors. The proposed method uses red-green-blue (RGB) images, six-axis robot pose and motor current data, gripper position and current data, as well as master position and orientation information as inputs without requiring force sensors. To estimate the interaction forces, a deep neural network composed of densely connected convolutional network (DenseNet) and long short-term memory (LSTM) is proposed. The database was created by operators using grip and picking motions to interact with 10 objects over a teleoperation system. In addition, we compared the proposed method with different deep learning networks that used different sets of inputs. The results show that the proposed model can estimate 1 kHz interaction force based on 60 Hz images and 1 kHz master inputs. Moreover, the results indicate that the master position and orientation information are useful in estimating the interaction force at a high sampling rate through the result of the change in the network input.Keywords
Funding Information
- National Research Foundation of Korea (NRF-2020R1A2C1008883)
This publication has 24 references indexed in Scilit:
- Inferring Interaction Force from Visual Information without Using Physical Force SensorsSensors, 2017
- Densely Connected Convolutional Networks2017
- Yale-CMU-Berkeley dataset for robotic manipulation researchThe International Journal of Robotics Research, 2017
- Using Physical Modeling and RGB-D Registration for Contact Force Sensing on Deformable Objects2017
- Towards Retrieving Force Feedback in Robotic-Assisted Surgery: A Supervised Neuro-Recurrent-Vision ApproachIEEE Transactions on Haptics, 2016
- Deep Residual Learning for Image Recognition2016
- Friction modeling with temperature effects for industrial robot manipulators2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- The Effect of Interaction Force Estimation on Performance in Bilateral TeleoperationIEEE Transactions on Haptics, 2011
- Long Short-Term MemoryNeural Computation, 1997