Empirical evaluation of fully Bayesian information criteria for mixture IRT models using NUTS
Open Access
- 17 June 2022
- journal article
- research article
- Published by Springer Nature in Behaviormetrika
- Vol. 50 (1) , 93-120
Abstract
This study is to evaluate the performance of fully Bayesian information criteria, namely, LOO, WAIC and WBIC in terms of the accuracy in determining the number of latent classes of a mixture IRT model while comparing it to the conventional model via non-random walk MCMC algorithms and to further compare their performance with conventional information criteria including AIC, BIC, CAIC, SABIC, and DIC. Monte Carlo simulations were carried out to evaluate these criteria under different situations. The results indicate that AIC, BIC, and their related CAIC and SABIC tend to select the simpler model and are not recommended when the actual data involve multiple latent classes. For the three fully Bayesian measures, WBIC can be used for detecting the number of latent classes for tests with at least 30 items, while WAIC and LOO are suggested to be used together with their effective number of parameters in choosing the correct number of latent classes.Keywords
This publication has 44 references indexed in Scilit:
- Markov chain Monte Carlo estimation of a mixture item response theory modelJournal of Statistical Computation and Simulation, 2013
- Parameter recovery and model selection in mixed Rasch modelsBritish Journal of Mathematical and Statistical Psychology, 2011
- Markov Chain Monte Carlo Estimation of Item Parameters for the Generalized Graded Unfolding ModelApplied Psychological Measurement, 2006
- Bayesian Measures of Model Complexity and FitJournal of the Royal Statistical Society Series B: Statistical Methodology, 2002
- Recovery of Item Parameters in the Nominal Response Model: A Comparison of Marginal Maximum Likelihood Estimation and Markov Chain Monte Carlo EstimationApplied Psychological Measurement, 2002
- Inference from Iterative Simulation Using Multiple SequencesStatistical Science, 1992
- Hybrid Monte CarloPhysics Letters B, 1987
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974
- The Monte Carlo MethodJournal of the American Statistical Association, 1949