Operator Expertise in Bilateral Teleoperation: Performance, Manipulation, and Gaze Metrics
Open Access
- 9 May 2025
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 14 (10) , 1923
Abstract
This paper presents a comprehensive user study aimed as assessing and differentiating operator expertise within bilateral teleoperation systems. The primary objective is to identify key performance metrics that effectively distinguish novice from expert users. Unlike prior approaches that focus primarily on psychological evaluations, this study emphasizes direct performance analysis across a range of telerobotic tasks. Ten participants (six novices and four experts) were assessed based on task completion time and difficulty, error rates, manipulator motion characteristics, gaze behaviour, and subjective feedback via questionnaires. The results show that experienced operators outperformed novices by completing tasks faster, making fewer errors, and demonstrating smoother manipulator control, as reflected by reduced jerks and higher spatial precision. Also, experts maintained consistent performance even as task complexity increased, whereas novices experienced a sharp decline, particularly at higher difficulty levels. Questionnaire responses further revealed that novices experienced higher mental and physical demands, especially in unfamiliar tasks, while experts demonstrated higher concentration and arousal levels. Additionally, the study introduces gaze transition entropy (GTE) and stationary gaze entropy (SGE) metrics to quantify visual attention strategies, with experts exhibiting more focused, goal-oriented gaze patterns, while novices showed more erratic and inefficient behaviour. These findings highlight both quantitative and qualitative measures as critical for evaluating operator performance and informing future teleoperation training programs.Keywords
Funding Information
- UK Research and Innovation (107463, EP/W006839/1)
- Nuclear Decommissioning Authority (NDA) (107463, EP/W006839/1)
- Tokyo Electric Power Company (TEPCO) (107463, EP/W006839/1)
- UKAEA/EPSRC Fusion (107463, EP/W006839/1)
This publication has 49 references indexed in Scilit:
- Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgeryInternational Journal of Computer Assisted Radiology and Surgery, 2023
- Eye gaze metrics for skill assessment and feedback in kidney stone surgeryInternational Journal of Computer Assisted Radiology and Surgery, 2023
- Objective assessment of robotic surgical skills: review of literature and future directionsSurgical Endoscopy, 2022
- The Emotional, Cognitive, Physiological, and Performance Effects of Variable Time Delay in Robotic TeleoperationInternational Journal of Social Robotics, 2017
- Effects of robotic manipulators on movements of novices and surgeonsSurgical Endoscopy, 2014
- Analysis of eye gaze: Do novice surgeons look at the same location as expert surgeons during a laparoscopic operation?Surgical Endoscopy, 2012
- Correlating motor performance with surgical error in laparoscopic cholecystectomySurgical Endoscopy, 2005
- Situation awareness and attention allocation measures for quantifying telepresence experiences in teleoperationHuman Factors and Ergonomics in Manufacturing & Service Industries, 2003
- Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workloadHuman Factors and Ergonomics in Manufacturing & Service Industries, 2000
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948