Post-hoc simulation study of computerized adaptive testing for the Korean Medical Licensing Examination
Open Access
- 17 May 2018
- journal article
- research article
- Published by Korea Health Personnel Licensing Examination Institute in Journal of Educational Evaluation for Health Professions
- Vol. 15, 14
Abstract
Purpose: Computerized adaptive testing (CAT) has been adopted in licensing examinations because it improves the efficiency and accuracy of the tests, as shown in many studies. This simulation study investigated CAT scoring and item selection methods for the Korean Medical Licensing Examination (KMLE). Methods: This study used a post-hoc (real data) simulation design. The item bank used in this study included all items from the January 2017 KMLE. All CAT algorithms for this study were implemented using the ‘catR’ package in the R program. Results: In terms of accuracy, the Rasch and 2-parametric logistic (PL) models performed better than the 3PL model. The ‘modal a posteriori’ and ‘expected a posterior’ methods provided more accurate estimates than maximum likelihood estimation or weighted likelihood estimation. Furthermore, maximum posterior weighted information and minimum expected posterior variance performed better than other item selection methods. In terms of efficiency, the Rasch model is recommended to reduce test length. Conclusion: Before implementing live CAT, a simulation study should be performed under varied test conditions. Based on a simulation study, and based on the results, specific scoring and item selection methods should be predetermined.Keywords
Funding Information
- Hallym University (201802-004)
- National Research Foundation of Korea (2017S1-A5B6053101)
This publication has 14 references indexed in Scilit:
- Presidential address: Preparing for permanent test centers and computerized adaptive testingJournal of Educational Evaluation for Health Professions, 2018
- Better Data From Better Measurements Using Computerized Adaptive TestingJournal of Methods and Measurement in the Social Sciences, 2011
- Bayesian Item Selection Criteria for Adaptive TestingPsychometrika, 1998
- Properties of Ability Estimation Methods in Computerized Adaptive TestingJournal of Educational Measurement, 1998
- Some New Item Selection Criteria for Adaptive TestingJournal of Educational and Behavioral Statistics, 1997
- Procedures for Selecting Items for Computerized Adaptive TestsApplied Measurement in Education, 1989
- Weighted Likelihood Estimation of Ability in Item Response TheoryPsychometrika, 1989
- Bias and Information of Bayesian Adaptive TestingApplied Psychological Measurement, 1984
- Adaptive EAP Estimation of Ability in a Microcomputer EnvironmentApplied Psychological Measurement, 1982
- A Broad-Range Tailored Test of Verbal AbilityApplied Psychological Measurement, 1977