Abstract
Background: This feasibility study describes the implementation of a self-directed mock oral examination (SD-MOE), which is administered in the absence of faculty examiners, and explores its potential as a supplemental tool in preparation for the American Board of Surgery Certifying Examination. Methods: During a multi-institutional MOE, all examinees completed the conventional MOE (C-MOE) and were presented the option to additionally participate in the SD-MOE. Rates of technical issues were compared between formats. Exploratory analyses evaluated participant satisfaction, resource intensity, exam performance metrics, and interrater reliability. Results: The C-MOE included 62 faculty examiners and 55 examinees, while the SD-MOE included 25 faculty examiners and 41 examinees. The frequency of technical issues did not differ between formats. Faculty examiner satisfaction was lower with the SD-MOE compared with the C-MOE (65% vs. 93%, P<0.01). Post-graduate year (PGY)-level correlated with mean case score (PGY-3: 2.45; PGY-4: 2.58; PGY-5: 2.70; P=0.01) and proportion of questions passed (PGY-3: 81.7%; PGY-4: 87.3%; PGY-5: 90.1%; P<0.01) in the C-MOE, but not the SD-MOE. PGY-level did not correlate with overall exam pass rates in either exam format. Interrater reliability did not differ significantly between the SD-MOE (weighted =0.32, 95% CI [0.19, 0.46]) and C-MOE (weighted =0.25, 95% CI [0.16, 0.35]). Conclusion: The SD-MOE was successfully administered with minimal logistical challenges. While limited by optional SD-MOE participation and differences in case counts between formats, this feasibility study suggests the SD-MOE can be considered as a supplemental tool for resource-limited institutions.