Exploring the Impact of Integrated AWE and Generative AI Feedback on Chinese EFL Undergraduates’ Higher-Order Thinking in Argumentative Writing
Open Access
- 7 January 2026
- journal article
- Published by SAGE Publications in Sage Open
- Vol. 16 (1)
Abstract
Although automated writing evaluation (AWE) and artificial intelligence (AI) tools have been widely practiced in EFL/ESL writing instruction, there is a lack of empirical research on the effect of the integration of both AWE and AI feedback on students’ higher-order thinking (HOT). Therefore, this study is to explore the impact of integrating AWE and AI feedback on Chinese EFL undergraduates’ higher-order thinking (HOT) in argumentative writing based on Revised Bloom’s Taxonomy and Cognitive Feedback Theory. Pre- and post-tests and semi-structured interviews were used to study 64 third-year students in the English major at a Chinese public university for 16 weeks. The experimental group ( n = 32) received AWE (Pigai) and AI (ChatGPT) feedback, while the control group ( n = 32) received only AWE (Pigai) feedback. Quantitative results showed that EG students had significant improvements in higher-order thinking (HOT; analysis, evaluation, and creation; p < .001) with a high effect size ( d > 0.80), while the CG students had a smaller improvement ( d > 0.15). ANOVA confirmed that analysis had the highest effect size ( p < .001, η 2 = .862), followed by evaluation ( p < .001, η 2 = .818) and creation ( p < .001, η 2 = .812). Qualitative results showed that AWE and AI tools were complementary, in which AWE could help students correct superficial language errors, but AI could improve students’ higher-order thinking (HOT) in analysis, evaluation, and creation. They can focus on language and higher-order thinking (HOT) and optimized revision strategies. However, students also faced problems in understanding feedback and over-reliance on it.Keywords
This publication has 27 references indexed in Scilit:
- Effects of an automated evaluation mechanism on students’ writing performance and higher-order thinking in an AR-based formative-peer-assessment learning modeEducation and Information Technologies, 2025
- Using generative ai as a simulation to support higher-order thinkingInternational Journal of Computer-Supported Collaborative Learning, 2024
- Artificial human thinking: ChatGPT’s capacity to be a model for critical thinking when prompted with problem-based writing activitiesDiscover Education, 2024
- A systematic review of AI-based automated written feedback researchReCALL, 2024
- Learner interaction with, and response to, AI-programmed automated writing evaluation feedback in EFL writing: An exploratory studyEducation and Information Technologies, 2023
- Enhancing argumentative writing with automated feedback and social comparison nudgingComputers & Education, 2022
- It takes two to tango: Investigating teacher-student interactions related to written corrective feedback with Activity TheoryAssessing Writing, 2022
- Development of syntactic complexity in Chinese university students’ L2 argumentative writingJournal of English for Academic Purposes, 2022
- L2 student engagement with automated feedback on writing: Potential for learning and issues of trustJournal of Second Language Writing, 2021
- Feedback on second language students' writingLanguage Teaching, 2006