The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments
Open Access
- 25 August 2020
- journal article
- research article
- Published by Wiley in Educational Measurement: Issues and Practice
- Vol. 40 (1) , 17-27
Abstract
Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.Keywords
This publication has 25 references indexed in Scilit:
- Linking item parameters to a base scaleAsia Pacific Education Review, 2011
- The Long-Term Sustainability of Different Item Response Theory Scaling MethodsEducational and Psychological Measurement, 2011
- A Comparison of Equating Methods and Linking Designs for Developing an Item Pool under Item Response TheoryBehaviormetrika, 2011
- An NCME Instructional Module on Booklet Designs in Large‐Scale Assessments of Student Achievement: Theory and PracticeEducational Measurement: Issues and Practice, 2009
- What PISA Did and Did Not Do: Germany after the ‘PISA-shock’European Educational Research Journal, 2009
- A Comparative Study of IRT Fixed Parameter Calibration MethodsJournal of Educational Measurement, 2006
- Educational standards and the changing discourse on education: the reception and consequences of the PISA study in GermanyOxford Review of Education, 2006
- Investigation of Student Growth Recovery in a Fixed-Item Linking Procedure With a Fixed-Person Prior Distribution for Mixed-Format Test DataApplied Measurement in Education, 2005
- Obtaining a Common Scale for Item Response Theory Item Parameters Using Separate Versus Concurrent Estimation in the Common-Item Equating DesignApplied Psychological Measurement, 2002
- A Rasch Model for Partial Credit ScoringPsychometrika, 1982