Towards Real-Time Physical Human-Robot Interaction Using Skeleton Information and Hand Gestures
- 1 October 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
For successful physical human-robot interaction, the capability of a robot to understand its environment is imperative. More importantly, the robot should extract from the human operator as much information as possible. A reliable 3D skeleton extraction is essential for a robot to predict the intentions of the operator while s/he moves toward the robot or performs a meaningful gesture. For this purpose, we have integrated a time-of-flight depth camera with a state-of-the-art 2D skeleton extraction library namely Openpose, to obtain 3D skeletal joint coordinates reliably. We have also developed a robust and rotation invariant (in the coronal plane)hand gesture detector using a convolutional neural network. At run time (after having been trained)the detector does not require any pre-processing of the hand images. A complete pipeline for skeleton extraction and hand gesture recognition is developed and employed for real-time physical human-robot interaction, demonstrating the promising capability of the designed framework. This work establishes a firm basis and will be extended for the development of intelligent human intention detection in physical human-robot interaction scenarios, to efficiently recognize a variety of static as well as dynamic gestures.Keywords
This publication has 12 references indexed in Scilit:
- Hand Keypoint Detection in Single Images Using Multiview Bootstrapping2017
- A human-robot interaction interface for mobile and stationary robots based on real-time 3D human body and hand-finger pose estimation2016
- A real-time Human-Robot Interaction system based on gestures for assistive scenariosComputer Vision and Image Understanding, 2016
- Convolutional Pose Machines2016
- Real-time human-robot interaction in complex environment using kinect v2 image recognition2015
- A Kinect-Based Gesture Recognition Approach for a Natural Human Robot InterfaceInternational Journal of Advanced Robotic Systems, 2015
- Hand gesture recognition for Human-Robot Interaction for service robot2012
- Gesture Recognition: A SurveyIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2007
- Learning methods for generic object recognition with invariance to pose and lighting2004
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998