Dimensionality in Compensatory MIRT When Complex Structure Exists: Evaluation of DETECT and NOHARM
- 2 September 2015
- journal article
- research article
- Published by Taylor & Francis in The Journal of Experimental Education
- Vol. 84 (2) , 398-420
Abstract
This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30% complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coefficients would decrease substantially.Keywords
This publication has 45 references indexed in Scilit:
- Generalized full-information item bifactor analysis.Psychological Methods, 2011
- Performance of DIMTEST- and NOHARM-Based Statistics for Testing UnidimensionalityApplied Psychological Measurement, 2007
- Comparison of NOHARM and DETECT in Item Cluster Recovery: Counting Dimensions and Allocating ItemsJournal of Educational Measurement, 2005
- Implications of Test Dimensionality for Unidimensional Irt Scoring: An Investigation of a High-Stakes Testing ProgramEducational and Psychological Measurement, 2000
- Assessing the Dimensionality of Item Response Matrices with Small Sample Sizes and Short Test LengthsApplied Measurement in Education, 1998
- The Multidimensional Random Coefficients Multinomial Logit ModelApplied Psychological Measurement, 1997
- Local Dependence Indexes for Item Pairs Using Item Response TheoryJournal of Educational and Behavioral Statistics, 1997
- Using Multidimensional Item Response Theory to Understand What Items and Tests Are MeasuringApplied Measurement in Education, 1994
- Thin versus Thick Matching in the Mantel-Haenszel Procedure for Detecting DIFJournal of Educational Statistics, 1993
- The Scientific Use of Factor Analysis in Behavioral and Life Sciences1978