Xiao Qunhan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xiao Qunhan has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
Bingwei Li
College of Foreign Studies, Hunan Normal University, Changsha, China
1 shared publication- Chunhua Mao
Foreign Studies College, Hunan Normal University, Changsha 410081, China
1 shared publication
Latest Publications
- Open Access
- 14 April 2026
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Differences between human and AI scoring: A meta-analysis of english language assessments
Despite a burgeoning body of research on using AI scoring systems in English assessments, concerns regarding their reliability persist. To fill this gap, this meta-analysis examined the AI-human scoring differences and the variables moderating these differences by synthesizing the results of 21 empirical studies with a total of 401,698 participants. Results indicate no statistically significant differences between AI and human scoring; the small effect size implies that the average systematic difference between the two was relatively modest. However, extremely high heterogeneity suggests that this overall finding masks considerable variability across study conditions. Moderator analyses reveal that AI-human scoring differences are significantly influenced by factors such as AI system type, number of human raters, agreement index employed, learner proficiency level (CEFR), and publication year. These findings suggest that while AI cannot fully replace human judgment, it can serve as a diagnostic reference tool within broader quality assurance frameworks. When significant discrepancies arise, they warrant investigation of both scoring sources. Based on these findings, this study offers evidence-based recommendations for educators on the effective use of AI scoring systems in language assessments.
- 14 April 2026
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Open AccessDifferences between human and AI scoring: A meta-analysis of english language assessments
Despite a burgeoning body of research on using AI scoring systems in English assessments, concerns regarding their reliability persist. To fill this gap, this meta-analysis examined the AI-human scoring differences and the variables moderating these differences by synthesizing the results of 21 empirical studies with a total of 401,698 participants. Results indicate no statistically significant differences between AI and human scoring; the small effect size implies that the average systematic difference between the two was relatively modest. However, extremely high heterogeneity suggests that this overall finding masks considerable variability across study conditions. Moderator analyses reveal that AI-human scoring differences are significantly influenced by factors such as AI system type, number of human raters, agreement index employed, learner proficiency level (CEFR), and publication year. These findings suggest that while AI cannot fully replace human judgment, it can serve as a diagnostic reference tool within broader quality assurance frameworks. When significant discrepancies arise, they warrant investigation of both scoring sources. Based on these findings, this study offers evidence-based recommendations for educators on the effective use of AI scoring systems in language assessments.