Digital Twin-Driven Mixed Reality Framework for Immersive Teleoperation With Haptic Rendering
- 18 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (12) , 8494-8501
Abstract
Teleoperation has widely contributed to many applications. Consequently, the design of intuitive and ergonomic control interfaces for teleoperation has become crucial. The rapid advancement of Mixed Reality (MR) has yielded tangible benefits in human-robot interaction. MR provides an immersive environment for interacting with robots, effectively reducing the mental and physical workload of operators during teleoperation. Additionally, the incorporation of haptic rendering, including kinaesthetic and tactile rendering, could further amplify the intuitiveness and efficiency of MR-based immersive teleoperation. In this study, we developed an immersive, bilateral teleoperation system, integrating Digital Twin-driven Mixed Reality (DTMR) manipulation with haptic rendering. This system comprises a commercial remote controller with a kinaesthetic rendering feature and a wearable cost-effective tactile rendering interface, called the Soft Pneumatic Tactile Array (SPTA). We carried out two user studies to assess the system's effectiveness, including a performance evaluation of key components within DTMR and a quantitative assessment of the newly developed SPTA. The results demonstrate an enhancement in both the human-robot interaction experience and teleoperation performance. For more project details, please view our website: https://sites.google.com/view/hbts-brl/homeKeywords
Funding Information
- Royal Society Research (RGS\R1\221122)
This publication has 22 references indexed in Scilit:
- A Wearable Soft Haptic Communicator Based on Dielectric Elastomer ActuatorsSoft Robotics, 2020
- Hamlyn CRM: a compact master manipulator for surgical robot remote controlInternational Journal of Computer Assisted Radiology and Surgery, 2020
- A Review on Mixed Reality: Current Trends, Challenges and ProspectsApplied Sciences, 2020
- A New Mixed-Reality-Based Teleoperation System for Telepresence and Maneuverability EnhancementIEEE Transactions on Human-Machine Systems, 2020
- Design and Verification of A Portable Master Manipulator Based on an Effective Workspace Analysis Framework2019
- A Handheld Master Controller for Robot-Assisted Microsurgery2019
- A microsurgical robot research platform for robot-assisted microsurgery research and trainingInternational Journal of Computer Assisted Radiology and Surgery, 2019
- Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and PerspectivesIEEE Transactions on Haptics, 2017
- State of the art of virtual reality technology2016
- Evaluation of the CyberGlove as a whole-hand input deviceACM Transactions on Computer-Human Interaction, 1995