Bayesian Comparison of Alternative Graded Response Models for Performance Assessment Applications
Authors:
- 22 March 2012
- journal article
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 72 (5) , 774-799
Abstract
This study examined the relative effectiveness of Bayesian model comparison methods in selecting an appropriate graded response (GR) model for performance assessment applications. Three popular methods were considered: deviance information criterion (DIC), conditional predictive ordinate (CPO), and posterior predictive model checking (PPMC). Using these methods, several alternative GR models were compared with Samejima’s unidimensional GR model, including the one-parameter GR model, the rating scale model, simple- and complex-structure two-dimensional GR models, and the GR model for testlets. Results from a simulation study indicated that these methods appeared to be equally accurate in selecting the preferred model. However, CPO and PPMC can be used to compare models at the item level, and PPMC can also be used to compare both the relative and absolute fit of different models.Keywords
This publication has 32 references indexed in Scilit:
- IRT Model Selection Methods for Dichotomous ItemsApplied Psychological Measurement, 2007
- Estimation of Compensatory and Noncompensatory Multidimensional Item Response Models Using Markov Chain Monte CarloApplied Psychological Measurement, 2003
- Bayesian Data Analysis2003
- Categorical Data Analysis2002
- MCMC Estimation and Some Model-Fit Analysis of Multidimensional IRT ModelsPsychometrika, 2001
- A Bayesian Random Effects Model for TestletsPsychometrika, 1999
- Local Dependence Indexes for Item Pairs Using Item Response TheoryJournal of Educational and Behavioral Statistics, 1997
- The Influence of Multidimensionality on the Graded Response ModelApplied Psychological Measurement, 1994
- A Predictive Approach to Model SelectionJournal of the American Statistical Association, 1979
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974