Corey Palermo
Research Subjects
Corey Palermo has a total of 60 co-authors. Here are the top co-authors by number of shared publications:
- Joshua Wilson
University of Delaware
8 shared publications Margareta Maria Thomson
North Carolina State University, USA
4 shared publications- Fan Zhang
University of Delaware
3 shared publications - Yue Huang
Measurement Incorporated Durham USA
3 shared publications - Andrew Potter
Learning Engineering Institute, Arizona State University, Interdisciplinary B, 1120 S. Cady Mall, Tempe, AZ 85281, USA
2 shared publications Arianto Wibowo
2 shared publications
Latest Publications
- 17 August 2026
- Published by Elsevier in Assessing Writing
- Vol. 70
AI and writing assessment: Innovations, validity, and ethical considerations
- 17 August 2026
- Published by Elsevier in Assessing Writing
- Vol. 70
AI and writing assessment: Innovations, validity, and ethical considerations
- 12 November 2025
- Published by American Educational Research Association (AERA) in American Educational Research Journal
- Vol. 63 (1) , 72-108
Impact of Automated Writing Evaluation (AWE) on Middle School Students’ Writing Outcomes
This study used a randomized, controlled trial to assess the impact of the automated writing evaluation system MI Write on middle school students from traditionally underserved groups, including Black and Hispanic/Latinx students and those experiencing poverty. The research focused on whether students who used MI Write for one school year (n = 1,260) showed improved argumentative writing performance, increased self-efficacy, greater enjoyment of writing, and stronger beliefs in the recursive nature of writing compared with those who did not use MI Write (n = 1,227) . No significant overall treatment effects were found, but significant treatment by district interactions emerged. Perceptions of MI Write were positive overall but varied by district. Findings reveal the limitations and promise of automated writing evaluation.
- 12 November 2025
- Published by American Educational Research Association (AERA) in American Educational Research Journal
- Vol. 63 (1) , 72-108
Impact of Automated Writing Evaluation (AWE) on Middle School Students’ Writing Outcomes
This study used a randomized, controlled trial to assess the impact of the automated writing evaluation system MI Write on middle school students from traditionally underserved groups, including Black and Hispanic/Latinx students and those experiencing poverty. The research focused on whether students who used MI Write for one school year (n = 1,260) showed improved argumentative writing performance, increased self-efficacy, greater enjoyment of writing, and stronger beliefs in the recursive nature of writing compared with those who did not use MI Write (n = 1,227) . No significant overall treatment effects were found, but significant treatment by district interactions emerged. Perceptions of MI Write were positive overall but varied by district. Findings reveal the limitations and promise of automated writing evaluation.
- Open Access
- 23 September 2025
- Published by Elsevier in Computers and Education: Artificial Intelligence
- Vol. 9
Identifying active ingredients and uptake patterns in the implementation of an AI-based writing support tool: Insights from a randomized controlled trial
- 23 September 2025
- Published by Elsevier in Computers and Education: Artificial Intelligence
- Vol. 9
Open AccessIdentifying active ingredients and uptake patterns in the implementation of an AI-based writing support tool: Insights from a randomized controlled trial
- Open Access
- 12 September 2025
- Published by Springer Nature in Reading and Writing
- Vol. 39 (3) , 1003-1032
Writing growth among non-English learners and English learners in grades 6–8 using automated writing evaluation
This study evaluates MI Write, an automated writing evaluation (AWE) system, as a benchmark for tracking middle-school students’ writing growth within multi-tiered systems of support. Multilevel growth models were applied to 3,299 students ( n = 464 English learners [ELs]) in Grades 6–8 across fall, winter, and spring benchmark writing prompts, estimating change in the MI Write Total Score, six analytic traits, and 81 NLP-derived features. Analyses compared whether growth differed according to EL-status measured as either a binary variable or as continuous measure of English language proficiency. Students gained an average of 1.6 points in overall quality across the school year, a standardized gain of d = 0.32. Non-ELs improved steadily, whereas ELs showed little growth between fall and winter but a steeper rise from winter to spring; total annual gains were equivalent, and English-proficiency did not predict growth. All traits improved, with ELs advancing slightly faster in development and sentence fluency. Feature analysis revealed ELs growing more in foundational skills (sentence complexity, spelling), while non-ELs progressed faster in advanced structures (subordination, sophisticated verbs, rare vocabulary). These distinct linguistic pathways produced similar overall gains, underscoring different instructional needs. Findings support MI Write’s viability as a formative benchmark that can inform data-driven decisions for diverse learners.
- 12 September 2025
- Published by Springer Nature in Reading and Writing
- Vol. 39 (3) , 1003-1032
Open AccessWriting growth among non-English learners and English learners in grades 6–8 using automated writing evaluation
This study evaluates MI Write, an automated writing evaluation (AWE) system, as a benchmark for tracking middle-school students’ writing growth within multi-tiered systems of support. Multilevel growth models were applied to 3,299 students ( n = 464 English learners [ELs]) in Grades 6–8 across fall, winter, and spring benchmark writing prompts, estimating change in the MI Write Total Score, six analytic traits, and 81 NLP-derived features. Analyses compared whether growth differed according to EL-status measured as either a binary variable or as continuous measure of English language proficiency. Students gained an average of 1.6 points in overall quality across the school year, a standardized gain of d = 0.32. Non-ELs improved steadily, whereas ELs showed little growth between fall and winter but a steeper rise from winter to spring; total annual gains were equivalent, and English-proficiency did not predict growth. All traits improved, with ELs advancing slightly faster in development and sentence fluency. Feature analysis revealed ELs growing more in foundational skills (sentence complexity, spelling), while non-ELs progressed faster in advanced structures (subordination, sophisticated verbs, rare vocabulary). These distinct linguistic pathways produced similar overall gains, underscoring different instructional needs. Findings support MI Write’s viability as a formative benchmark that can inform data-driven decisions for diverse learners.
- Open Access
- 27 August 2025
- Published by National Council on Measurement in Education (NCME) in Chinese/english Journal of Educational Measurement and Evaluation
- Vol. 6, 5
An Early Review of Generative Language Models in Automated Writing Evaluation: Advancements, Challenges, and Future Directions for Automated Essay Scoring and Feedback Generation
Automated writing evaluation (AWE) has long supported assessment and instruction, yet existing systems struggle to capture deeper rhetorical and pedagogical aspects of student writing. Recent advances in generative language models (GLMs) such as GPT and Llama present new opportunities, but their effectiveness remains uncertain. This review synthesizes 29 studies on automated essay scoring and 14 on automated writing feedback generation, examining how GLMs are applied through prompting, fine-tuning, and adaptation. Findings show GLMs can approximate human scoring and deliver richer, rubric-aligned feedback, but fairness, validity, and ethical issues remain largely unaddressed. We conclude that GLMs hold promise to enhance AWE, provided that future work establishes robust evaluation frameworks and safeguards to ensure responsible, equitable use.
- 27 August 2025
- Published by National Council on Measurement in Education (NCME) in Chinese/english Journal of Educational Measurement and Evaluation
- Vol. 6, 5
Open AccessAn Early Review of Generative Language Models in Automated Writing Evaluation: Advancements, Challenges, and Future Directions for Automated Essay Scoring and Feedback Generation
Automated writing evaluation (AWE) has long supported assessment and instruction, yet existing systems struggle to capture deeper rhetorical and pedagogical aspects of student writing. Recent advances in generative language models (GLMs) such as GPT and Llama present new opportunities, but their effectiveness remains uncertain. This review synthesizes 29 studies on automated essay scoring and 14 on automated writing feedback generation, examining how GLMs are applied through prompting, fine-tuning, and adaptation. Findings show GLMs can approximate human scoring and deliver richer, rubric-aligned feedback, but fairness, validity, and ethical issues remain largely unaddressed. We conclude that GLMs hold promise to enhance AWE, provided that future work establishes robust evaluation frameworks and safeguards to ensure responsible, equitable use.
- 1 June 2025
- Published by National Council on Measurement in Education (NCME) in Chinese/english Journal of Educational Measurement and Evaluation
- Vol. 6, 6
生成式语言模型在自动写作评估中的早期综述:自动作文评分与反馈生成领域的进展、挑战及方向
自动写作评估(AWE)长期以来在写作测评与教学中发挥着重要作用,但现有系统在捕捉学生写作中更深层次的修辞结构与教学相关特征方面仍存在不足。近年来,生成式语言模型(GLM)如 GPT 和 Llama 的快速发展为 AWE带来新机遇,但其效能尚未得到充分验证。本综述综合了 29 项关于自动作文评分的研究和 14 项关于自动写作反馈生成的研究,系统探讨了 GLM 在提示工程、微调与适应性策略中的应用。研究表明,GLM 在模拟人类评分与生成符合评分标准的反馈方面展现出潜力,但公平性、效度与伦理等关键问题仍亟待解决。GLM 有望显著提升 AWE 系统的表现,但前提是未来研究需建构稳健的评估框架与保障机制,以确保其在教育实践中负责任、公平地应用。
- 1 June 2025
- Published by National Council on Measurement in Education (NCME) in Chinese/english Journal of Educational Measurement and Evaluation
- Vol. 6, 6
生成式语言模型在自动写作评估中的早期综述:自动作文评分与反馈生成领域的进展、挑战及方向
自动写作评估(AWE)长期以来在写作测评与教学中发挥着重要作用,但现有系统在捕捉学生写作中更深层次的修辞结构与教学相关特征方面仍存在不足。近年来,生成式语言模型(GLM)如 GPT 和 Llama 的快速发展为 AWE带来新机遇,但其效能尚未得到充分验证。本综述综合了 29 项关于自动作文评分的研究和 14 项关于自动写作反馈生成的研究,系统探讨了 GLM 在提示工程、微调与适应性策略中的应用。研究表明,GLM 在模拟人类评分与生成符合评分标准的反馈方面展现出潜力,但公平性、效度与伦理等关键问题仍亟待解决。GLM 有望显著提升 AWE 系统的表现,但前提是未来研究需建构稳健的评估框架与保障机制,以确保其在教育实践中负责任、公平地应用。
- 1 May 2024
- Published by Taylor & Francis
- p. 23-39
Automated Essay Evaluation at Scale
This chapter presents a progressive human–machine hybrid scoring approach designed to address limitations of automated scoring applications in large-scale, summative assessment programs. The chapter begins with context about modern summative assessment scoring programs, including current stakeholder expectations and constraints. Next, the Project Essay Grade (PEG) automated scoring engine is described, along with a summary of evidence for its use. Engine training and validation requirements in support of the progressive hybrid scoring approach are then proposed, followed by a discussion of the training process, including fairness considerations. Next, the deployment of PEG by Measurement Incorporated in a multi-state summative assessment context is described to illustrate the approach. The chapter concludes with implications for the field and future directions.
- 1 May 2024
- Published by Taylor & Francis
- p. 23-39
Automated Essay Evaluation at Scale
This chapter presents a progressive human–machine hybrid scoring approach designed to address limitations of automated scoring applications in large-scale, summative assessment programs. The chapter begins with context about modern summative assessment scoring programs, including current stakeholder expectations and constraints. Next, the Project Essay Grade (PEG) automated scoring engine is described, along with a summary of evidence for its use. Engine training and validation requirements in support of the progressive hybrid scoring approach are then proposed, followed by a discussion of the training process, including fairness considerations. Next, the deployment of PEG by Measurement Incorporated in a multi-state summative assessment context is described to illustrate the approach. The chapter concludes with implications for the field and future directions.
- 1 May 2024
- Published by Taylor & Francis
- p. 261-277
Relationships between Middle-School Teachers' Perceptions and Application of Automated Writing Evaluation and Student Performance
This study uses mixed methods to explore relations among middle school teachers’ (n = 19) perceptions of the automated writing evaluation (AWE) system MI Write and their system usage and students’ writing performance. We used a convergent parallel design that involved concurrent quantitative and qualitative strands of data collection. Findings indicated that teachers perceived MI Write as easy to us, but there was greater variation in their perceptions of MI Write’s usefulness and social desirability, particularly across school districts. Additionally, although teachers had generally perceived MI Write positively, none of the teachers implemented the system sufficiently to meet our overall fidelity expectations. Nevertheless, between two-thirds and three-quarters of teachers demonstrated aligned perceptions and usage, via reporting high perceptions and exhibiting high usage or low perceptions and low usage. Finally, based on a novel data visualization, we identified a trend in which perceptions and usage of MI Write were higher among teachers whose students made significant gains in writing during the school year than teachers whose students made no gains or decreased in writing performance from baseline to follow-up. However, the trend was not dramatic and there were exceptions. Implications for AWE implementation, research, and development are discussed.
- 1 May 2024
- Published by Taylor & Francis
- p. 261-277
Relationships between Middle-School Teachers' Perceptions and Application of Automated Writing Evaluation and Student Performance
This study uses mixed methods to explore relations among middle school teachers’ (n = 19) perceptions of the automated writing evaluation (AWE) system MI Write and their system usage and students’ writing performance. We used a convergent parallel design that involved concurrent quantitative and qualitative strands of data collection. Findings indicated that teachers perceived MI Write as easy to us, but there was greater variation in their perceptions of MI Write’s usefulness and social desirability, particularly across school districts. Additionally, although teachers had generally perceived MI Write positively, none of the teachers implemented the system sufficiently to meet our overall fidelity expectations. Nevertheless, between two-thirds and three-quarters of teachers demonstrated aligned perceptions and usage, via reporting high perceptions and exhibiting high usage or low perceptions and low usage. Finally, based on a novel data visualization, we identified a trend in which perceptions and usage of MI Write were higher among teachers whose students made significant gains in writing during the school year than teachers whose students made no gains or decreased in writing performance from baseline to follow-up. However, the trend was not dramatic and there were exceptions. Implications for AWE implementation, research, and development are discussed.
- 13 February 2024
- Published by Elsevier in Learning and Instruction
- Vol. 91
Elementary English learners’ engagement with automated feedback
- 13 February 2024
- Published by Elsevier in Learning and Instruction
- Vol. 91
Elementary English learners’ engagement with automated feedback
- Open Access
- 26 December 2023
- Published by Elsevier in Computers & Education
- Vol. 211, 104985
Predictors of middle school students’ perceptions of automated writing evaluation
- 26 December 2023
- Published by Elsevier in Computers & Education
- Vol. 211, 104985
Open AccessPredictors of middle school students’ perceptions of automated writing evaluation