Cheating in OSCEs: The Impact of Simulated Security Breaches on OSCE Performance
Authors:
- 7 September 2016
- journal article
- research article
- Published by Taylor & Francis in Teaching and Learning in Medicine
- Vol. 29 (1) , 52-58
Abstract
Construct: Valid score interpretation is important for constructs in performance assessments such as objective structured clinical examinations (OSCEs). An OSCE is a type of performance assessment in which a series of standardized patients interact with the student or candidate who is scored by either the standardized patient or a physician examiner. Background: In high-stakes examinations, test security is an important issue. Students accessing unauthorized test materials can create an unfair advantage and lead to examination scores that do not reflect students' true ability level. The purpose of this study was to assess the impact of various simulated security breaches on OSCE scores. Approach: Seventy-six 3rd-year medical students participated in an 8-station OSCE and were randomized to either a control group or to 1 of 2 experimental conditions simulating test security breaches: station topic (i.e., providing a list of station topics prior to the examination) or egregious security breach (i.e., providing detailed content information prior to the examination). Overall total scores were compared for the 3 groups using both a one-way between-subjects analysis of variance and a repeated measure analysis of variance to compare the checklist, rating scales, and oral question subscores across the three conditions. Results: Overall total scores were highest for the egregious security breach condition (81.8%), followed by the station topic condition (73.6%), and they were lowest for the control group (67.4%). This trend was also found with checklist subscores only (79.1%, 64.9%, and 60.3%, respectively for the security breach, station topic, and control conditions). Rating scale subscores were higher for both the station topic and egregious security breach conditions compared to the control group (82.6%, 83.1%, and 77.6%, respectively). Oral question subscores were significantly higher for the egregious security breach condition (88.8%) followed by the station topic condition (64.3%), and they were the lowest for the control group (48.6%). Conclusions: This simulation of different OSCE security breaches demonstrated that student performance is greatly advantaged by having prior access to test materials. This has important implications for medical educators as they develop policies and procedures regarding the safeguarding and reuse of test content.Keywords
This publication has 19 references indexed in Scilit:
- Ethical Academic Judgments and Behaviors: Applying a Multidimensional Ethics Scale to Measure the Ethical Academic Behavior of Graduate StudentsEthics & Behavior, 2012
- The Impact of Repeat Information on Examinee Performance for a Large-Scale Standardized-Patient ExaminationAcademic Medicine, 2010
- Was a breach of examination security unfair in an objective structured clinical examination? A critical incidentMedical Teacher, 2003
- Cheating in Academic Institutions: A Decade of ResearchEthics & Behavior, 2001
- Are "tomorrow's doctors" honest? Questionnaire study exploring medical students' attitudes and reported behaviour on academic misconductBMJ, 2001
- Modeling the Effects of a Test Security Breach on a Large-scale Standardized Patient Examination with a Sample of International Medical GraduatesAcademic Medicine, 2000
- Modeling the effects of security breaches on studentsʼ performances on a large-scale standardized patient examinationAcademic Medicine, 1999
- Cheating in medical schoolAcademic Medicine, 1996
- Is test security a concern when OSCE stations are repeated across clerkship rotations?Academic Medicine, 1996
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960