An Investigation of Sample Size Splitting on ATFIND and DIMTEST
- 26 February 2013
- journal article
- research article
- Published by SAGE Publications in Educational and Psychological Measurement
- Vol. 73 (4) , 631-647
Abstract
Modeling multidimensional test data with a unidimensional model can result in serious statistical errors, such as bias in item parameter estimates. Many methods exist for assessing the dimensionality of a test. The current study focused on DIMTEST. Using simulated data, the effects of sample size splitting for use with the ATFIND procedure for empirically deriving a subtest composed of items that potentially measure a second dimension versus DIMTEST for assessing whether this subtest represents a second dimension were investigated. Conditions explored included proportion of sample used for ATFIND, sample size, test length, interability correlations, test structure, and distribution of item difficulties. Overall, it appears that DIMTEST has Type I error rates near the nominal rate and good power in detecting multidimensionality, although Type I error inflation is observed for larger sample sizes. Results suggest that a 50/50 split maximizes power and keeps the Type I error rate below the nominal level unless the test is short and the sample is large. A 75/25 split controls Type I error better for short tests and large samples.Keywords
This publication has 28 references indexed in Scilit:
- A Bootstrap Generalization of Modified Parallel Analysis for IRT Dimensionality AssessmentApplied Measurement in Education, 2008
- Conditional Covariance-Based Subtest Selection for DIMTESTApplied Psychological Measurement, 2008
- Comparison of Factor Simplicity Indices for Dichotomous Data: DETECT R, Bentler's Simplicity Index, and the Loading Simplicity IndexApplied Measurement in Education, 2008
- Statistical Versus Substantive DimensionalityEducational and Psychological Measurement, 2006
- Detecting Multidimensionality Due to Curricular DifferencesJournal of Educational Measurement, 2003
- The Performance of Dimtest When Latent Trait and Item Difficulty Distributions DifferApplied Psychological Measurement, 2000
- Assessing Dimensionality of a Set of Item Responses‐Comparison of Different ApproachesJournal of Educational Measurement, 1994
- Multidimensionality and Item Bias in Item Response TheoryApplied Psychological Measurement, 1992
- The Difficulty of Test Items That Measure More Than One AbilityApplied Psychological Measurement, 1985
- An Examination of the Characteristics of Unidimensional IRT Parameter Estimates Derived From Two-Dimensional DataApplied Psychological Measurement, 1985