GradeDrift-LLM: Measuring Student-History-Induced Score Drift in LLM-Based Automated Grading
Authors:
- Catalin AnghelCatalin Anghel
- Andreea Alexandra AnghelAAAndreea Alexandra Anghel
- Marian Viorel CraciunMCMarian Viorel Craciun
- Adina CocuACAdina Cocu
- Simona MoldovanuSimona Moldovanu
- Cristian SanduCSCristian Sandu
- Christiana Diana Maria DragosloveanuCDChristiana Diana Maria Dragosloveanu
- Constantin Adrian AndreiCAConstantin Adrian Andrei
- Diana-Elena VulpeDVDiana-Elena Vulpe
- Calina MaierCMCalina Maier
- Cristian ScheauCristian Scheau
- Serban DragosloveanuSerban Dragosloveanu
Open Access
- 21 August 2026
- journal article
- research article
- Published by MDPI AG in Machine Learning and Knowledge Extraction
- Vol. 8 (8) , 252
Abstract
Background: Large language models (LLMs) are increasingly explored for automated educational assessment, while future educational platforms may combine grading, feedback, learner analytics, and personalization. The objective of this study was to determine whether student-history metadata can influence the numerical score assigned to the same answer. Methods: This study introduces GradeDrift-LLM, a controlled framework for measuring student-history-induced score drift in LLM-based automated grading. We evaluated 1000 Computer Science answers from 100 students across six student-history conditions and eight open-weight LLMs. For each grading instance, the submitted answer, question, reference answer, rubric-related information, scoring scale, and grading instruction were kept constant; only the student-history condition varied. Results: Across 39,997 valid paired comparisons, 83.92% showed no drift, 9.40% showed upward drift, and 6.68% showed downward drift. Mean absolute drift was 0.2137 points, and the 95th percentile absolute drift was 1 point. Positive-history frames tended to increase scores, whereas negative-history frames tended to decrease them. Drift was model-dependent, not uniformly explained by approximate scale, and present in both technical and argumentative answers; rare extreme deviations reached 10 points. Conclusions: Student-history metadata can influence LLM-generated grading scores despite explicit instructions to ignore it. Future LLM-based grading systems should separate answer-based scoring from learner-context-based personalization and validate score invariance under controlled learner-context variations.Keywords
Funding Information
- University of Medicine and Pharmacy Carol Davila
This publication has 29 references indexed in Scilit:
- A survey on LLM-as-a-judgeThe Innovation, 2026
- Learner models: design, components, structure, and modellingUser Modeling and User-Adapted Interaction, 2025
- Evaluating large language models as raters in large-scale writing assessments: A psychometric framework for reliability and validityComputers and Education: Artificial Intelligence, 2025
- Is GPT-4 fair? An empirical analysis in automatic short answer gradingComputers and Education: Artificial Intelligence, 2025
- Grading explanations of problem-solving process and generating feedback using large language models at human-level accuracyPhysical Review Physics Education Research, 2025
- Evaluating large language models for criterion-based grading from agreement to consistencynpj Science of Learning, 2024
- Development and techniques in learner model in adaptive e-learning system: A systematic reviewComputers & Education, 2024
- Grading exams using large language models: A comparison between human and AI grading of exams in higher education using ChatGPTBritish Educational Research Journal, 2024
- Effects of Rubrics on Academic Performance, Self-Regulated Learning, and self-Efficacy: a Meta-analytic ReviewEducational Psychology Review, 2023
- The use of scoring rubrics for formative assessment purposes revisited: A reviewEducational Research Review, 2013