Detecting Test Tampering Using Item Response Theory
- 23 January 2015
- journal article
- research article
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 75 (6) , 931-953
Abstract
Test tampering, especially on tests for educational accountability, is an unfortunate reality, necessitating that the state (or its testing vendor) perform data forensic analyses, such as erasure analyses, to look for signs of possible malfeasance. Few statistical approaches exist for detecting fraudulent erasures, and those that do largely do not lend themselves to making probabilistic statements about the likelihood of the observations. In this article, a new erasure detection index, EDI, is developed, which uses item response theory to compare the number of observed wrong-to-right erasures to the number expected due to chance, conditional on the examinee’s ability-level and number of erased items. A simulation study is presented to evaluate the Type I error rate and power of EDI under various types of fraudulent and benign erasures. Results show that EDI with a correction for continuity yields Type I error rates that are less than or equal to nominal levels for every condition studied, and has high power to detect even small amounts of tampering among the students for whom tampering is most likely.Keywords
This publication has 6 references indexed in Scilit:
- Modeling Answer Changes on Test ItemsJournal of Educational and Behavioral Statistics, 2012
- Detecting Answer Copying When the Regular Response Process Follows a Known Response ModelJournal of Educational and Behavioral Statistics, 2006
- Test Scoring2001
- Detection of Answer Copying with Unknown Item and Trait ParametersApplied Psychological Measurement, 1998
- A Nominal Response Model Approach for Detecting Answer CopyingApplied Psychological Measurement, 1997
- Marginal Maximum Likelihood Estimation of Item Parameters: Application of an EM AlgorithmPsychometrika, 1981