A Method for Identifying Partial Test-Taking Engagement
Authors:
- 1 March 2021
- journal article
- research article
- Published by Taylor & Francis in Applied Measurement in Education
- Vol. 34 (2) , 150-161
Abstract
Effort-moderated (E-M) scoring is intended to estimate how well a disengaged test taker would have performed had they been fully engaged. It accomplishes this adjustment by excluding disengaged responses from scoring and estimating performance from the remaining responses. The scoring method, however, assumes that the remaining responses are not affected by disengagement. A recent study provided evidence that the responses used in E-M scoring can sometimes reflect less-than-full, or partial engagement. In these instances, E-M scores will be less trustworthy because they can still be distorted by disengagement. This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.Keywords
This publication has 25 references indexed in Scilit:
- Using response time to investigate students' test-taking behaviors in a NAEP computer-based studyLarge-scale Assessments in Education, 2014
- An Investigation of Examinee Test-Taking Effort on a Large-Scale AssessmentApplied Measurement in Education, 2013
- A model of examinee test-taking effort.2011
- Correlates of Rapid-Guessing Behavior in Low-Stakes Testing: Implications for Test Development and Measurement PracticeApplied Measurement in Education, 2009
- Changes in Rapid-Guessing Behavior Over a Series of AssessmentsEducational Assessment, 2007
- An Application of Item Response Time: The Effort‐Moderated IRT ModelJournal of Educational Measurement, 2006
- Response Time Effort: A New Measure of Examinee Motivation in Computer-Based TestsApplied Measurement in Education, 2005
- Construct‐Irrelevant Variance in High‐Stakes TestingEducational Measurement: Issues and Practice, 2004
- Appropriateness measurement with polychotomous item response models and standardized indicesBritish Journal of Mathematical and Statistical Psychology, 1985
- THE PSYCHOLOGY OF EDUCATIONAL MEASUREMENTJournal of Educational Measurement, 1984