Classification Accuracy of Assigning Student Performance to Proficiency Levels: Guidelines for Assessment Design
Authors:
- Kadriye ErcikanKEKadriye Ercikan
- Marc JulianMJMarc Julian
- 1 July 2002
- journal article
- Published by Taylor & Francis in Applied Measurement in Education
- Vol. 15 (3) , 269-294
Abstract
In this study we examined the degree of variation in the accuracy of classifying student performance to proficiency-level scores with changes in the number of proficiency levels and the measurement accuracy. Furthermore, we examined the degree to which the classification accuracy varies across different ability levels given different numbers of proficiency levels based on the same test and the same set of cut-scores. The results of the study based on simulations indicate that the classification accuracy decreased, on average, by 10% for an increase of 1 proficiency level, 20% for an increase of 2 proficiency levels, and 20% to 30% for an increase of 3 proficiency levels. In addition, classification accuracy varied 10% to 20% for tests with reliabilities that ranged between 0.70 and 0.93. The findings regarding the variability of classification accuracy for different score ranges point to serious limitations of interpretability of single indexes that are intended to represent classification accuracy. Sugge...Keywords
This publication has 16 references indexed in Scilit:
- Reliability of Credentialing Examinations and the Impact of Scoring Models and Standard-Setting PoliciesApplied Measurement in Education, 1997
- Estimating the Consistency and Accuracy of Classifications Based on Test ScoresJournal of Educational Measurement, 1995
- A Generalized Partial Credit Model: Application of an EM AlgorithmApplied Psychological Measurement, 1992
- Developing a Common Metric in Item Response TheoryApplied Psychological Measurement, 1983
- Marginal Maximum Likelihood Estimation of Item Parameters: Application of an EM AlgorithmPsychometrika, 1981
- ASSESSING THE RELIABILITY OF TESTS USED TO MAKE PASS/FAIL DECISIONSJournal of Educational Measurement, 1979
- ON THE RELIABILITY OF DECISIONS IN DOMAIN‐REFERENCED TESTINGJournal of Educational Measurement, 1976
- Estimating Item Parameters and Latent Ability when Responses are Scored in Two or More Nominal CategoriesPsychometrika, 1972
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951