Diagnosing Chinese EFL learners’ writing ability using polytomous cognitive diagnostic models
- 26 May 2023
- journal article
- research article
- Published by SAGE Publications in Language Testing
- Vol. 41 (1) , 109-134
Abstract
Cognitive diagnostic assessment (CDA) intends to identify learners’ strengths and weaknesses in latent cognitive attributes to provide personalized remedial instructions. Previous CDA studies on English as a Foreign Language (EFL)/English as a Second Language (ESL) writing have adopted dichotomous cognitive diagnostic models (CDMs) to analyze data from checklists using simple yes/no judgments. Compared to descriptors with multiple levels, descriptors with only yes/no judgments were considered too absolute, potentially resulting in misjudgment of learners’ writing ability. However, few studies have used polytomous CDMs to analyze graded response data from rating scales to diagnose writing ability. This study applied polytomous CDMs to diagnose 1166 EFL learners’ writing performance scored with a three-level rating scale. The sG-DINA model was selected after comparing model-data fit statistics of multiple polytomous CDMs. The results of classification accuracy indices and item discrimination indices further demonstrated that sG-DINA had good performance on identifying learners’ strengths and weaknesses. The generated diagnostic information at group and individual levels was further synthesized into a personalized diagnostic report, although its usefulness still requires further investigation. The findings provided evidence for the feasibility of applying polytomous CDM in EFL writing assessment.Keywords
Funding Information
- National Office for Philosophy and Social Sciences (17BYY015)
This publication has 38 references indexed in Scilit:
- Diagnosing EAP writing ability using the Reduced Reparameterized Unified ModelLanguage Testing, 2011
- The Generalized DINA Model FrameworkPsychometrika, 2011
- Rater types in writing performance assessments: A classification approach to rater variabilityLanguage Testing, 2008
- Measurement of psychological disorders using cognitive diagnosis models.Psychological Methods, 2006
- Higher-Order Latent Trait Models for Cognitive DiagnosisPsychometrika, 2004
- Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response TheoryApplied Psychological Measurement, 2001
- Theoretical Perspectives on WritingAnnual Review of Applied Linguistics, 1998
- Explanatory Variables for EFL Students' Expository WritingLanguage Learning, 1996
- Toward an Understanding of the Distinct Nature of L2 Writing: The ESL Research and Its ImplicationsTESOL Quarterly, 1993
- RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORYJournal of Educational Measurement, 1983