Predictive Validity of Virtual Reality Simulator Metrics for Operating Room Performance Across Surgical Specialties: A Systematic Review (Preprint)

Authors:
Preprint
Publisher website
Google Scholar
Abstract
BACKGROUND Virtual reality (VR) simulators record objective metrics such as task time, path length and errors, and programs across surgical specialties increasingly consider using them for progression and credentialing. Evidence that VR training improves operative performance does not establish that an individual's simulator score predicts how that person will operate on patients. OBJECTIVE This systematic review aimed to determine whether objective VR simulator metrics predict externally assessed operating room (OR) performance of surgeons and surgical trainees across surgical specialties. METHODS We searched PubMed/MEDLINE, Scopus and Web of Science from inception to 15 September 2026 and searched citations. Eligible English-language studies enrolled surgeons or surgical trainees in any surgical specialty and reported an individual-level association between a VR simulator metric and rated or timed performance during surgery on human patients, or a patient outcome of that surgery. Records excluded under an earlier general surgery-only definition were rescreened. Risk of bias was appraised with the Quality In Prognosis Studies (QUIPS) tool, and findings were synthesized narratively following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020. The review was not prospectively registered. RESULTS Of 2902 records screened and 170 full texts assessed, 23 studies with 360 analyzed participants were included (one study did not report its sample size): seven in general surgery, seven in urology, five in ophthalmology, two in gynecology, one in otolaryngology and one of unknown specialty. Associations were predominantly supportive in 13 studies, mixed in four, null or inverse in five and equivocal in one. Four of five ophthalmology studies were supportive: Eyesi scores correlated with masked ratings of cataract surgery (r=0.65; P=.003), and in one program first-year scores correlated with third-year operative ratings (r=0.81; P=.002). In urology, the strongest correlations came from samples that pooled trainees and experts (rho=0.81; P<.001), whereas correlations within experts were weak (rho=0.37; P=.18) and absent in the study of unknown specialty (r=-0.03). Two prostatectomy studies related VR suturing metrics to continence, in both directions. In general surgery and gynecology, associations were weak, inconsistent or inverse. Risk of bias was moderate in 11 studies and high in 12, and nine were appraised from abstracts alone. CONCLUSIONS VR metrics correlate with operative performance most clearly when samples span a wide range of experience, when the same instrument scores both performances and when the simulated task reproduces the operation. These conditions overstate its ability to discriminate among trainees at a similar stage. The evidence is small, mostly single-center and largely concurrent. A stand-alone VR score should not be used for high-stakes competency or credentialing decisions; VR assessment belongs within multimodal, longitudinal assessment. CLINICALTRIAL The review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement; the completed checklist is provided in Multimedia Appendix 1. It was not registered prospectively with the International Prospective Register of Systematic Reviews (PROSPERO), and no separate protocol was prepared.

This publication has 35 references indexed in Scilit: