A Simulation Study to Compare Nonequivalent Groups With Anchor Test Equating and Pseudo‐Equivalent Group Linking
Authors:
- 28 February 2018
- journal article
- research article
- Published by Wiley in ETS Research Report Series
- Vol. 2018 (1) , 1-16
Abstract
In this paper we compare the newly developed pseudo‐equivalent groups (PEG) linking method with the linking methods based on the traditional nonequivalent groups with anchor test (NEAT) design and illustrate how to use the PEG methods under imperfect equating conditions. To do this, we proposed a new method that combines the features of PEG linking and NEAT equating (referred as PEGAT) and compared it with NEAT and PEG. PEG mainly uses test takers' background variables to create PEG and then links scores on different forms whereas NEAT equating adjusts group differences in ability through the anchor test scores. The proposed method, PEGAT, uses background variables and anchor scores to adjust group ability differences. Using simulated data, these 3 linking methods were compared in 2 equating scenarios: small and large group difference in ability. The simulation design was based on real data on a test in operation. The test scores and the background variables were assumed to have a multivariate multinomial distribution. A log linear model was used to manipulate and produce simulated data. For PEG and PEGAT linking, 3 different sets of background variables were manipulated to study the impact of correlation strength of background variables to the total scores. Our results showed that NEAT linking outperformed PEG linking when the group ability difference was large, but that NEAT linking could be improved by incorporating the PEG adjustment procedure based on background variables. When the groups were similar in ability, PEG linking produced comparable results to NEAT. This finding justifies the use of PEG linking when a good anchor test is not available as well as the use of PEGAT when a good anchor is available but needs to be strengthened by background variables.Keywords
This publication has 18 references indexed in Scilit:
- Observed Score Linear Equating with CovariatesJournal of Educational Measurement, 2011
- Consequences of Violated Equating Assumptions Under the Equivalent Groups DesignInternational Journal of Testing, 2011
- Accumulative Equating Error after a Chain of Linear EquatingsPsychometrika, 2010
- Matching Methods for Causal Inference: A Review and a Look ForwardStatistical Science, 2010
- Estimating Comparable Scores Using Surrogate VariablesApplied Psychological Measurement, 2001
- Univariate and Bivariate Loglinear Models for Discrete Test Score DistributionsJournal of Educational and Behavioral Statistics, 2000
- Effect on Equating Results of Matching Samples on an Anchor TestApplied Measurement in Education, 1990
- What Combination of Sampling and Equating Methods Works Best?Applied Measurement in Education, 1990
- Sensitivity of Equating Results to Different Sampling StrategiesApplied Measurement in Education, 1990
- Adjustment by Minimum Discriminant InformationThe Annals of Statistics, 1984