Combining Propensity Scores and Common Items for Test Score Equating
Open Access
- 30 July 2025
- journal article
- Published by SAGE Publications in Applied Psychological Measurement
- Vol. 50 (3) , 115-144
Abstract
Ensuring that test scores are fair and comparable across different test forms and different test groups is a significant statistical challenge in educational testing. Methods to achieve score comparability, a process known as test score equating, often rely on including common test items or assuming that test taker groups are similar in key characteristics. This study explores a novel approach that combines propensity scores, based on test takers’ background covariates, with information from common items using kernel smoothing techniques for binary-scored test items. An empirical analysis using data from a high-stakes college admissions test evaluates the standard errors and differences in adjusted test scores. A simulation study examines the impact of factors such as the number of test takers, the number of common items, and the correlation between covariates and test scores on the method’s performance. The findings demonstrate that integrating propensity scores with common item information reduces standard errors and bias more effectively than using either source alone. This suggests that balancing the groups on the test-takers’ covariates enhance the fairness and accuracy of test score comparisons across different groups. The proposed method highlights the benefits of considering all the collected data to improve score comparability.Keywords
Funding Information
- Swedish Wallenberg MMW (2019.0129)
This publication has 38 references indexed in Scilit:
- Efficiency Analysis of Item Response Theory Kernel Equating for Mixed-Format TestsApplied Psychological Measurement, 2023
- Evaluating Equating Transformations in IRT Observed-Score and Kernel Equating MethodsApplied Psychological Measurement, 2022
- How Important is the Choice of Bandwidth in Kernel Equating?Applied Psychological Measurement, 2021
- Effect of Statistically Matching Equating Samples for Common‐Item EquatingETS Research Report Series, 2021
- Linking With External Covariates: Examining Accuracy by Anchor Type, Test Length, Ability Difference, and Sample SizeApplied Psychological Measurement, 2019
- A Simulation Study to Compare Nonequivalent Groups With Anchor Test Equating and Pseudo‐Equivalent Group LinkingETS Research Report Series, 2018
- Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studiesStatistics in Medicine, 2015
- Kernel Equating Under the Non-Equivalent Groups With Covariates DesignApplied Psychological Measurement, 2015
- A comparison of statistical selection strategies for univariate and bivariate log‐linear modelsBritish Journal of Mathematical and Statistical Psychology, 2010
- Goodness‐of‐fit diagnostics for the propensity score model when estimating treatment effects using covariate adjustment with the propensity scorePharmacoepidemiology and Drug Safety, 2008