Empirical Power and Type I Error Rates for an IRT Fit Statistic that Considers the Precision of Ability Estimates
Author:
- 1 August 2003
- journal article
- other
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 63 (4) , 566-583
Abstract
Model-data-fit of item response theory (IRT) models is generally assessed by comparing observed performance by examinees on individual items with performance that is predicted under the chosen IRT model. However, use of traditional chi-square methods to evaluate goodness-of-fit of IRT models is not appropriate when the underlying trait/ability is estimated imprecisely (e.g., shorter assessments). This article describes a goodness-of-fit statistic that considers directly the uncertainty with which ability is estimated as well as a resampling-based hypothesis testing procedure. A simulation study was conducted to evaluate the empirical power and Type I error rates for the proposed procedure. Results of the study indicated that the procedure should be useful for evaluating goodness-of-fit of IRT models for most testing applications where uncertainty in ability estimation is an issue.Keywords
This publication has 13 references indexed in Scilit:
- The Effect of Errors in Estimating Ability on Goodness-of-Fit Tests for Irt ModelsEducational and Psychological Measurement, 2000
- Monte Carlo Based Null Distribution for an Alternative Goodness‐of‐Fit Test Statistic in IRT ModelsJournal of Educational Measurement, 2000
- Item Response Theory and Classical Test Theory: An Empirical Comparison of their Item/Person StatisticsEducational and Psychological Measurement, 1998
- The Conceptual Framework for the Development of a Mathematics Performance Assessment InstrumentEducational Measurement: Issues and Practice, 1993
- A Comparison of Item- and Person-Fit Methods of Assessing Model-Data Fit in IRTApplied Psychological Measurement, 1990
- The Effect of Uncertainty of Item Parameter Estimation on Ability EstimatesPsychometrika, 1990
- A Comparison of Several Goodness-of-Fit StatisticsApplied Psychological Measurement, 1985
- Using Simulation Results to Choose a Latent Trait ModelApplied Psychological Measurement, 1981
- A Method for Simulating Non-Normal DistributionsPsychometrika, 1978
- Estimating Item Parameters and Latent Ability when Responses are Scored in Two or More Nominal CategoriesPsychometrika, 1972