Developing individualized feedback for listening assessment: Combining standard setting and cognitive diagnostic assessment approaches
Authors:
- 25 February 2021
- journal article
- research article
- Published by SAGE Publications in Language Testing
- Vol. 39 (1) , 90-116
Abstract
In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3,358 students’ item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL learners. The results showed that proficiency classifications and subskill mastery classifications were generally of acceptable reliability, and the two kinds of classifications were in alignment with each other at individual and group levels. The outcome of the study is a set of descriptors that describe each test taker’s ability to understand certain level of oral texts and his or her cognitive performance. The current study, by illustrating the feasibility of combining standard setting and CDA approaches to produce individualized feedback, contributes to the enhancement of score reporting and addresses the long-standing criticism that large-scale language assessments fail to provide individualized feedback to link assessment with instruction.Keywords
Funding Information
- National Social Science Foundation of China (19CYY050 & 20BYY107)
This publication has 66 references indexed in Scilit:
- When Does Scale Anchoring Work? A Case StudyJournal of Educational Measurement, 2011
- How Often Do Subscores Have Added Value? Results from Operational and Simulated DataJournal of Educational Measurement, 2010
- Investigating the decision-making process of standard setting participantsLanguage Testing, 2010
- Assessing the accuracy and consistency of language proficiency classification under competing measurement modelsLanguage Testing, 2010
- Proficiency descriptors based on a scale-anchoring study of the new TOEFL iBT reading testLanguage Testing, 2007
- On the consistency of individual classification using short scales.Psychological Methods, 2007
- Application of the rule-space procedure to language testing: examining attributes of a free response listening testLanguage Testing, 1998
- Estimating the Consistency and Accuracy of Classifications Based on Test ScoresJournal of Educational Measurement, 1995
- An Investigation of Classification Consistency Indexes Estimated Under Alternative Strong True Score ModelsJournal of Educational Measurement, 1990
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978