Abstract
Box trainers are widely used for laparoscopic skills training due to their ease of setup and realistic tool-tissue interaction. Automating skill assessment on these systems can enhance training by reducing analysis time and eliminating subjective evaluations. Achieving this requires accurate real-time estimation of the instrument tooltip’s three-dimensional (3D) pose. However, existing methods either depend on additional sensing hardware, which introduces ergonomic challenges, or lack real-time capabilities. A modular architecture has been developed for estimation of an instrument’s tooltip pose from video frames acquired during laparoscopic training. Our method incorporates a GPU-based algorithm that takes input from a customized deep learning model (YOLOv7), detects the edges of the surgical instrument, and computes tooltip’s 3D pose. Additionally, an automated 3D-pose dataset generation technique is introduced to generate datasets for training the model. Our method was evaluated using a vision-based tracking experiment, achieving 3D positional errors of 0.72 mm, 0.88 mm, and 3.81 mm over the X, Y, and Z-axes, and an orientation error of 5.65° with an end-to-end latency of 144 ms and sustained real-time throughput of 11.4 frames per second (fps). The proposed method enables accurate estimation of laparoscopic instrument tooltip poses without the need for additional hardware or markers. This work represents a significant step toward automated skill assessment in laparoscopic training using box trainers.
Funding Information
  • Qatar Research Development and Innovation Council ‘‘QRDI’’ Academic Research (ARG01-0430-230047)
  • Graduate Sponsorship Research Award (GSRA11-L-1-0319-24006)