L2 writers’ critical AI literacy in AI-assisted academic writing: scale development and validation
Authors:
Top Cited Papers
- 17 March 2026
- journal article
- research article
- Published by Walter de Gruyter GmbH in International Review of Applied Linguistics in Language Teaching
- Vol. 64 (4) , 2005-2031
Abstract
The proliferation of generative AI (GenAI) in academic writing necessitates that second language (L2) writers develop Critical AI Literacy (CAIL) – a set of competencies extending beyond operational skills to include ethical, strategic, and critical engagement. While conceptual frameworks for CAIL are emerging, a validated instrument to empirically measure this multifaceted construct for L2 writers has been conspicuously absent. This study addresses this gap by detailing the development and validation of the L2 Writers’ Critical AI Literacy Scale (L2-CAILS). Following established psychometric procedures, the scale was developed through literature review, expert validation, and pilot testing, and was subsequently administered to 420 L2 university students. The data were subjected to both exploratory (EFA) and confirmatory factor analysis (CFA) to establish construct validity and reliability. The analysis yielded a robust and reliable 25-item, five-factor model of CAIL: (1) Awareness and Understanding of AI, (2) Ethical and Responsible Use, (3) AI Application Strategies, (4) Critical Evaluation of AI Outputs, and (5) Metacognitive Engagement and Self-Regulation. The L2-CAILS provides a validated tool for researchers and educators to assess L2 writers’ competencies, design targeted pedagogical interventions, and conduct empirical research on the role of CAIL in AI-assisted academic writing.Keywords
This publication has 45 references indexed in Scilit:
- Teaching EFL students to write with ChatGPT: Students' motivation to learn, cognitive load, and satisfaction with the learning processEducation and Information Technologies, 2024
- Exploring Students’ Generative AI-Assisted Writing Processes: Perceptions and Experiences from Native and Nonnative English SpeakersTechnology, Knowledge and Learning, 2024
- Impacts of ChatGPT-assisted writing for EFL English majors: Feasibility and challengesEducation and Information Technologies, 2024
- Collaborating with ChatGPT in argumentative writing classroomsAssessing Writing, 2023
- Using ChatGPT for second language writing: Pitfalls and potentialsAssessing Writing, 2023
- Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigationEducation and Information Technologies, 2023
- Developing a model for AI Across the curriculum: Transforming the higher education landscape via innovation in AI literacyComputers and Education: Artificial Intelligence, 2023
- Conceptualizing AI literacy: An exploratory reviewComputers and Education: Artificial Intelligence, 2021
- Language Learning Online as a Social PracticeTESOL Quarterly, 2013
- The content validity index: Are you sure you know what's being reported? critique and recommendationsResearch in Nursing & Health, 2006