Using Markov chain model to evaluate medical students’ trajectory on progress tests and predict USMLE step 1 scores---a retrospective cohort study in one medical school
Authors:
Open Access
- 9 April 2021
- journal article
- research article
- Published by Springer Nature in BMC Medical Education
- Vol. 21 (1) , 1-9
Abstract
Background: Medical students must meet curricular expectations and pass national licensing examinations to become physicians. However, no previous studies explicitly modeled stages of medical students acquiring basic science knowledge. In this study, we employed an innovative statistical model to characterize students’ growth using progress testing results over time and predict licensing examination performance. Methods: All students matriculated from 2016 to 2017 in our medical school with USMLE Step 1 test scores were included in this retrospective cohort study (N = 358). Markov chain method was employed to: 1) identify latent states of acquiring scientific knowledge based on progress tests and 2) estimate students’ transition probabilities between states. The primary outcome of this study, United States Medical Licensing Examination (USMLE) Step 1 performance, were predicted based on students’ estimated probabilities in each latent state identified by Markov chain model. Results: Four latent states were identified based on students’ progress test results: Novice, Advanced Beginner I, Advanced Beginner II and Competent States. At the end of the first year, students predicted to remain in the Novice state had lower mean Step 1 scores compared to those in the Competent state (209, SD = 14.8 versus 255, SD = 10.8 respectively) and had more first attempt failures (11.5% versus 0%). On regression analysis, it is found that at the end of the first year, if there was 10% higher chance staying in Novice State, Step 1 scores will be predicted 2.0 points lower (95% CI: 0.85–2.81 with P < .01); while 10% higher chance in Competent State, Step 1scores will be predicted 4.3 points higher (95% CI: 2.92–5.19 with P < .01). Similar findings were also found at the end of second year medical school. Conclusions: Using the Markov chain model to analyze longitudinal progress test performance offers a flexible and effective estimation method to identify students’ transitions across latent stages for acquiring scientific knowledge. The results can help identify students who are at-risk for licensing examination failure and may benefit from targeted academic support.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Relationship Between Performance on the NBME Comprehensive Basic Sciences Self-Assessment and USMLE Step 1 for U.S. and Canadian Medical School StudentsAcademic Medicine, 2010
- Validity of Four Approaches of Using Repeatersʼ MCAT Scores in Medical School Admissions to Predict USMLE Step 1 Total ScoresAcademic Medicine, 2010
- An Introduction to Latent Class Growth Analysis and Growth Mixture ModelingSocial and Personality Psychology Compass, 2007
- A Stage-Sequential Model of Reading Transitions: Evidence From the Early Childhood Longitudinal Study.Journal of Educational Psychology, 2005
- AIC and BICSociological Methods & Research, 2004
- Multiple Imputation for Missing DataSociological Methods & Research, 2000
- Using covariance structure analysis to detect correlates and predictors of individual change over time.Psychological Bulletin, 1994
- Latent Curve AnalysisPsychometrika, 1990
- Stochastic Modeling of Elementary Psychological ProcessesThe American Journal of Psychology, 1985
- A growth curve approach to the measurement of change.Psychological Bulletin, 1982