Validation of an Online Assessment of Orthopedic Surgery Residents' Cognitive Skills and Preparedness for Carpal Tunnel Release Surgery
Authors:
- Janet ShanedlingJSJanet Shanedling
- Ann Van HeestAHAnn Van Heest
- Michael RodriguezMRMichael Rodriguez
- Matthew PutnamMPMatthew Putnam
- Julie AgelJAJulie Agel
- 1 September 2010
- journal article
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 2 (3) , 435-441
Abstract
Background: Surgical competence requires both cognitive and technical skills. Relatively little is found in the literature regarding the value of Web-based assessments to measure surgery residents' mastery of the competencies. Objective: To determine the validity and reliability of 2 online instruments for predicting the cognitive preparedness of residents for performing carpal tunnel release surgery. Method: Twenty-eight orthopedic residents and 2 medical school students responded to an online measure of their perception of preparedness and to an online cognitive skills assessment prior to an objective structured assessment of technical skills, in which they performed carpal tunnel release surgery on cadaveric specimens and received a pass/fail assessment. The 2 online assessments were analyzed for their internal reliability, external correlation with the pass/fail decision, and construct validity. Results: The internal consistency of the perception of preparedness measure was high (α = .92) while the cognitive assessment was less strong (α = .65). Both instruments demonstrated moderately strong correlations with the pass/fail decision, with Spearman correlation of .606 ( P = .000) and .617 ( P = .000), respectively. Using logistic regression to analyze the predictive strength of each instrument, the perception of preparedness measure demonstrated a 76% probability (η 2 = .354) and the cognitive skills assessment a 73% probability (η 2 = .381) of correctly predicting the pass/fail decision. Analysis of variance modeling resulted in significant differences between levels at P < .005, supporting good construct validity. Conclusions: The online perception of preparedness measure and the cognitive skills assessment both are valid and reliable predictors of readiness to successfully pass a cadaveric motor skills test of carpal tunnel release surgery.Keywords
This publication has 14 references indexed in Scilit:
- Resident Self-Assessment and Self-Reflection: University of Wisconsin-Madison’s Five-Year StudyJournal of General Internal Medicine, 2009
- Evaluating the competency of gynecology residents in the operating room: validation of a new assessment toolAmerican Journal of Obstetrics and Gynecology, 2008
- An Evaluation of the Feasibility, Validity, and Reliability of Laparoscopic Skills Assessment in the Operating RoomAnnals of Surgery, 2007
- Teaching Surgical Skills — Changes in the WindNew England Journal of Medicine, 2006
- Assessing and Documenting General Competencies in Otolaryngology Resident Training ProgramsThe Laryngoscope, 2006
- Self-assessment of resident surgical skills: Is it feasible?American Journal of Obstetrics and Gynecology, 2005
- Assessment of resident surgical skills: Is testing feasible?American Journal of Obstetrics and Gynecology, 2005
- Web-based minimally invasive surgery training: competency assessment in PGY 1-2 surgical residentsCurrent Surgery, 2004
- Objective structured assessment of technical skill (OSATS) for surgical residentsBritish Journal of Surgery, 1997
- Objective structured assessment of technical skill (OSATS) for surgical residentsBritish Journal of Surgery, 1997