Using GPT‐4 to Augment Imbalanced Data for Automatic Scoring
Authors:
Open Access
- 19 November 2025
- journal article
- Published by Wiley in Journal of Educational Measurement
- Vol. 62 (4) , 959-995
Abstract
Machine learning‐based automatic scoring faces challenges with imbalanced student responses across scoring categories. To address this, we introduce a novel text data augmentation framework that leverages GPT‐4, a generative large language model specifically tailored for imbalanced datasets in automatic scoring. Our experimental dataset consisted of student‐written responses to four science items. We crafted prompts for GPT‐4 to generate responses, especially for minority scoring classes, enhancing the dataset. We then fine‐tuned DistilBERT for automatic scoring based on the augmented and original datasets. Model performance was assessed using accuracy, precision, recall, and F 1 metrics. Our findings revealed that incorporating GPT‐4‐augmented data significantly improved model performance, particularly in terms of precision and F 1 scores. Interestingly, the extent of improvement varied depending on the specific dataset and the proportion of augmented data used. Notably, we found that a varying amount of augmented data (20%‐40%) was required to achieve stable improvement in automatic scoring. Comparisons with models trained on additional student‐written responses suggest that GPT‐4 augmented models align with those trained on student data. This research highlights the potential and effectiveness of data augmentation techniques, utilizing generative large language models like GPT‐4, in addressing imbalanced datasets within automatic assessment.Keywords
All Related Versions
Funding Information
- National Science Foundation (2100964)
- National Science Foundation (2101104)
- National Science Foundation (2101112)
- National Science Foundation (2101166)
This publication has 36 references indexed in Scilit:
- Role play with large language modelsNature, 2023
- Effectiveness of data augmentation to predict students at risk using deep learning algorithmsSocial Network Analysis and Mining, 2023
- Data augmentation approaches in natural language processing: A surveyAI Open, 2022
- Data Augmentation for Enlarging Student Feature Space and Improving Random Forest Success Prediction2021
- Knowledge Distillation: A SurveyInternational Journal of Computer Vision, 2021
- A Meta-Analysis of Machine Learning-Based Science Assessments: Factors Impacting Machine-Human Score AgreementsJournal of Science Education and Technology, 2020
- Comparison of Machine Learning Performance Using Analytic and Holistic Coding Approaches Across Constructed Response Assessments Aligned to a Science Learning ProgressionJournal of Science Education and Technology, 2020
- The Potential for the Use of Deep Neural Networks in e-Learning Student Evaluation with New Data Augmentation Method2020
- AUG-BERT: An Efficient Data Augmentation Algorithm for Text Classification2020
- Conditional BERT Contextual Augmentation2019