Simultaneous Estimation of Overall Score and Subscores Using MIRT , HO-IRT and Bi-factor Model on TIMSS Data
- 1 March 2020
- journal article
- research article
- Published by Egitimde ve Psikolojide Olcme ve Degerlendirme Dergisi in Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi
- Vol. 11 (1) , 61-75
Abstract
In educational testing, there is an increasing interest in the simultaneous estimation of the overall scores and subscores. This study aims to compare the reliability and precision of the simultaneous estimation of overall scores and sub-scores using MIRT, HO-IRT and Bi-factor models. TIMSS 2015 mathematics scores have been used as a data set in this study. The TIMSS 2015 mathematics test consists of 35 items, four of which are polytomously scored (0-1-2), and the rest of the items are dichotomously scored (0-1). The four content domains include number (14 items), algebra (9 items), geometry (6 items), and data and change (6 items). Ability parameters were estimated using the BMIRT software. The results showed that the MIRT and HO-IRT methods performed similarly in terms of precision and reliability for subscore estimates. The MIRT maximum information method had the smallest standard error of measurement for the overall score estimates. All three methods performed similarly in terms of the overall score reliability. The findings suggest that among the three methods compared, HO-IRT appears to be a better choice in the simultaneous estimation of the overall score and subscores for the data from TIMSS 2015. Recommendations for the testing practices and future research are provided.Keywords
This publication has 31 references indexed in Scilit:
- A Tutorial on Interpreting Bifactor Model ScoresInternational Journal of Testing, 2013
- Generalized full-information item bifactor analysis.Psychological Methods, 2011
- Parameter Estimation With Small Sample Size A Higher-Order IRT Model ApproachApplied Psychological Measurement, 2010
- When Can Subscores Have Value?Journal of Educational and Behavioral Statistics, 2008
- A Multidimensional Item Response Modeling Approach for Improving Subscale Proficiency Estimation and ClassificationApplied Psychological Measurement, 2007
- A Multidimensional Partial Credit Model With Associated Item and Test Statistics: An Application to Mixed-Format TestsApplied Psychological Measurement, 2006
- Improving Measurement Precision of Test Batteries Using Multidimensional Item Response Models.Psychological Methods, 2004
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesStructural Equation Modeling: A Multidisciplinary Journal, 1999
- How Is Reliability Related to the Quality of Test Scores? What Is the Effect of Local Dependence on Reliability?Educational Measurement: Issues and Practice, 1996
- The Difficulty of Test Items That Measure More Than One AbilityApplied Psychological Measurement, 1985