A Real-Time Markerless Approach for 3D Pose Estimation of Laparoscopic Instruments in Surgical Training Simulators
Open Access
- 24 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 216646-216662
Abstract
Box trainers are widely used for laparoscopic skills training due to their ease of setup and realistic tool-tissue interaction. Automating skill assessment on these systems can enhance training by reducing analysis time and eliminating subjective evaluations. Achieving this requires accurate real-time estimation of the instrument tooltip’s three-dimensional (3D) pose. However, existing methods either depend on additional sensing hardware, which introduces ergonomic challenges, or lack real-time capabilities. A modular architecture has been developed for estimation of an instrument’s tooltip pose from video frames acquired during laparoscopic training. Our method incorporates a GPU-based algorithm that takes input from a customized deep learning model (YOLOv7), detects the edges of the surgical instrument, and computes tooltip’s 3D pose. Additionally, an automated 3D-pose dataset generation technique is introduced to generate datasets for training the model. Our method was evaluated using a vision-based tracking experiment, achieving 3D positional errors of 0.72 mm, 0.88 mm, and 3.81 mm over the X, Y, and Z-axes, and an orientation error of 5.65° with an end-to-end latency of 144 ms and sustained real-time throughput of 11.4 frames per second (fps). The proposed method enables accurate estimation of laparoscopic instrument tooltip poses without the need for additional hardware or markers. This work represents a significant step toward automated skill assessment in laparoscopic training using box trainers.Keywords
Funding Information
- Qatar Research Development and Innovation Council ‘‘QRDI’’ Academic Research (ARG01-0430-230047)
- Graduate Sponsorship Research Award (GSRA11-L-1-0319-24006)
This publication has 61 references indexed in Scilit:
- Objective evaluation of laparoscopic surgical skills in wet lab training based on motion analysis and machine learningLangenbeck's Archives of Surgery, 2022
- Electromagnetic tracking in image‐guided laparoscopic surgery: Comparison with optical tracking and feasibility study of a combined laparoscope and laparoscopic ultrasound systemMedical Physics, 2018
- Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery2017
- Automated robot‐assisted surgical skill evaluation: Predictive analytics approachThe International Journal of Medical Robotics and Computer Assisted Surgery, 2017
- Real-Time 3D Tracking of Articulated Tools for Robotic Surgery2016
- 2D/3D Real-Time Tracking of Surgical Instruments Based on Endoscopic Image Processing2016
- Image Based Surgical Instrument Pose Estimation with Multi-class Labelling and Optical Flow2015
- Microsoft COCO: Common Objects in Context2014
- 2D-3D Pose Tracking of Rigid Instruments in Minimally Invasive Surgery2014
- Construct validation of the ProMIS simulator using a novel laparoscopic suturing taskSurgical Endoscopy, 2005