DIF Detection for Multiple Groups: Comparing Three‐Level GLMMs and Multiple‐Group IRT Models
Open Access
- 14 February 2024
- journal article
- research article
- Published by Wiley in Journal of Educational Measurement
- Vol. 61 (2) , 325-344
Abstract
For assessment scales applied to different groups (e.g., students from different states; patients in different countries), multigroup differential item functioning (MG‐DIF) needs to be evaluated in order to ensure that respondents with the same trait level but from different groups have equal response probabilities on a particular item. The current study compares two approaches for DIF detection: a multiple‐group item response theory (MG‐IRT) model and a generalized linear mixed model (GLMM). In the MG‐IRT model approach, item parameters are constrained to be equal across groups and DIF is evaluated for each item in each group. In the GLMM, groups are treated as random, and item difficulties are modeled as correlated random effects with a joint multivariate normal distribution. Its nested structure allows the estimation of item difficulty variances and covariances at the group level. We use an excerpt from the PISA 2015 reading domain as an exemplary empirical investigation, and conduct a simulation study to compare the performance of the two approaches. Results from the empirical investigation show that the detection of countries with DIF is similar in both approaches. Results from the simulation study confirm this finding and indicate slight advantages of the MG‐IRT model approach.Keywords
This publication has 18 references indexed in Scilit:
- Deep Neural Networks for Detecting Statistical Model Misspecifications. The Case of Measurement InvarianceStructural Equation Modeling: A Multidisciplinary Journal, 2022
- An Examination of Measurement Invariance Using a Multilevel Explanatory Rasch ModelMeasurement: Interdisciplinary Research and Perspectives, 2020
- What makes the difference? The impact of item properties on mode effects in reading assessmentsStudies in Educational Evaluation, 2019
- A Monte Carlo Simulation Study to Assess The Appropriateness of Traditional and Newer Approaches to Test for Measurement InvarianceStructural Equation Modeling: A Multidisciplinary Journal, 2019
- Evaluating item response theory linking and model fit for data from PISA 2000–2012Assessment in Education: Principles, Policy & Practice, 2018
- Multiple-Group Factor Analysis AlignmentStructural Equation Modeling: A Multidisciplinary Journal, 2014
- Classification accuracy and cut point selectionStatistics in Medicine, 2012
- A Generalized Logistic Regression Procedure to Detect Differential Item Functioning Among Multiple GroupsInternational Journal of Testing, 2011
- Relaxing Measurement Invariance in Cross-National Consumer Research Using a Hierarchical IRT ModelJournal of Consumer Research, 2007
- Weighted Likelihood Estimation of Ability in Item Response TheoryPsychometrika, 1989