Comparing the Weighted Gain Score and a Rasch-Based Approach for Estimating Learning Outcomes in Medical Education: Quantitative Study
Open Access
- 16 June 2026
- journal article
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 12 (1) , e75516
Abstract
Background: Pretest-posttest designs are widely used to estimate learning gain in studies evaluating educational interventions in medical education. The Weighted Gain Score (WGS) was proposed to reduce bias associated with differences in baseline performance. Objective: This study evaluated the statistical and inferential properties of the WGS by comparing it to Rasch Learning Gain (RLG) across 3 datasets. Methods: The WGS implements a weighting coefficient that includes the parameter µ, which linearly rescales the difference between pretest and posttest percentage scores. We examined the effect of varying µ (30, 50, and 70) on learning gain calculations and compared the results with those obtained using RLG. The following three datasets were analyzed: (1) a small illustrative dataset demonstrating the mathematical behavior of the WGS, (2) an empirical dataset from a previous educational evaluation study, and (3) a randomly generated binomial dataset designed to examine the metric under larger sample conditions. Results: Changing the parameter µ in the WGS affected the magnitude of the calculated learning gains: lower µ-values produced larger gain estimates, whereas higher µ-values produced smaller estimates. Despite these differences in scale, the WGS and RLG correlated strongly in both the empirical dataset ( r =0.93; P <.001) and the simulated dataset ( r =0.92; P <.001); variation in µ did not alter the inferential results. Both methods identified the same interaction effect in the empirical dataset. Conclusions: The WGS produced results highly consistent with those of RLG while requiring substantially lower computational complexity. The metric can be applied to both small and large datasets and allows µ to function as an adjustment coefficient for calibrating learning gain estimates across cohorts without altering inferential conclusions.Keywords
This publication has 46 references indexed in Scilit:
- Sensitivity Analysis in Meta‐Analysis: A TutorialCochrane Evidence Synthesis and Methods, 2026
- Working more collaboratively to better understand teaching and its quality: Challenges faced and possible solutionsStudies in Educational Evaluation, 2021
- Zero‐cell corrections in random‐effects meta‐analysesResearch Synthesis Methods, 2020
- Digital Affinity in Medical Students Influences Learning Outcome: A Cluster Analytical Design Comparing Vodcast With Traditional LectureJournal of Surgical Education, 2019
- Spearman–Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applicationsJournal of Clinical Epidemiology, 2017
- How feedback can foster professional growth of teachers in the clinical workplace: A review of the literatureStudies in Educational Evaluation, 2015
- Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank dataQuality of Life Research, 2013
- Measuring learning gain during a one-day introductory bronchoscopy courseSurgical Endoscopy, 2010
- Reflections on experimental research in medical educationAdvances in Health Sciences Education, 2008
- What to add to nothing? Use and avoidance of continuity corrections in meta‐analysis of sparse dataStatistics in Medicine, 2004