Evaluation of single-stage vision models for pose estimation of surgical instruments
- 30 April 2023
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 18 (12) , 2125-2142
Abstract
Purpose Multiple applications in open surgical environments may benefit from adoption of markerless computer vision depending on associated speed and accuracy requirements. The current work evaluates vision models for 6-degree of freedom pose estimation of surgical instruments in RGB scenes. Potential use cases are discussed based on observed performance. Methods Convolutional neural nets were developed with simulated training data for 6-degree of freedom pose estimation of a representative surgical instrument in RGB scenes. Trained models were evaluated with simulated and real-world scenes. Real-world scenes were produced by using a robotic manipulator to procedurally generate a wide range of object poses. Results CNNs trained in simulation transferred to real-world evaluation scenes with a mild decrease in pose accuracy. Model performance was sensitive to input image resolution and orientation prediction format. The model with highest accuracy demonstrated mean in-plane translation error of 13 mm and mean long axis orientation error of 5\(^{\circ }\) in simulated evaluation scenes. Similar errors of 29 mm and 8\(^{\circ }\) were observed in real-world scenes. Conclusion 6-DoF pose estimators can predict object pose in RGB scenes with real-time inference speed. Observed pose accuracy suggests that applications such as coarse-grained guidance, surgical skill evaluation, or instrument tracking for tray optimization may benefit from markerless pose estimation.
Keywords
This publication has 112 references indexed in Scilit:
- Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted displayJournal of Biomedical Informatics, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Assessing the Magnitude and Costs of Intraoperative Inefficiencies Attributable to Surgical Instrument TraysJournal of the American College of Surgeons, 2014
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Validating a Global Rating Scale to Monitor Individual Resident Learning Curves During Arthroscopic Knee Meniscal RepairArthroscopy, 2013
- Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes2013
- Objective structured assessment of technical skills and checklist scales reliability compared for high stakes assessmentsAnz Journal of Surgery, 2012
- Observational tools for assessment of procedural skills: a systematic reviewThe American Journal of Surgery, 2011
- Metrics for 3D Rotations: Comparison and AnalysisJournal of Mathematical Imaging and Vision, 2009
- Objective structured assessment of technical skill (OSATS) for surgical residentsBritish Journal of Surgery, 1997