Hannah Horne-Robinson
Hannah Horne-Robinson has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Owen Henkel
University of Oxford
2 shared publications - Baptiste Moreau-Pernet
Digital Harbor Foundation
1 shared publication - Bill Roberts
Legible Labs, New York, USA
1 shared publication - Greg Thompson
TangibleAI
1 shared publication - Joshua McGrane
University of Melbourne, Assessment & Evaluation Research Centre
1 shared publication - Kirk Vanacore
Cornell University, Worcester, NY, USA
1 shared publication
Latest Publications
- Open Access
- 23 March 2025
- Published by Society for Learning Analytics Research in Journal of Learning Analytics
- Vol. 12 (1) , 50-64
Learning to Love LLMs for Answer Interpretation
This paper introduces AMMORE, a new dataset of 53,000 math open-response question-answer pairs from Rori, a mathematics learning platform used by middle and high school students in several African countries. Using this dataset, we conducted two experiments to evaluate the use of large language models (LLM) for grading particularly challenging student answers. In experiment 1, we use a variety of LLM-driven approaches, including zero-shot, few-shot, and chain-of-thought prompting, to grade the 1% of student answers that a rule-based classifier fails to grade accurately. We find that the best-performing approach — chain-of-thought prompting — accurately scored 97% of these edge cases, effectively boosting the overall accuracy of the grading from 96% to 97%. In experiment 2, we aim to better understand the consequential validity of the improved grading accuracy by passing grades generated by the best-performing LLM-based approach to a Bayesian Knowledge Tracing (BKT) model, which estimated student mastery of specific lessons. We find that modest improvements in model accuracy can lead to significant changes in mastery estimation. Where the rule-based classifier misclassified the mastery status of 6.9% of students across completed lessons, using the LLM chain-of-thought approach reduced this to 2.6%. These findings suggest that LLMs could be valuable for grading fill-in questions in mathematics education, potentially enabling wider adoption of open-response questions in learning systems.
- 23 March 2025
- Published by Society for Learning Analytics Research in Journal of Learning Analytics
- Vol. 12 (1) , 50-64
Open AccessLearning to Love LLMs for Answer Interpretation
This paper introduces AMMORE, a new dataset of 53,000 math open-response question-answer pairs from Rori, a mathematics learning platform used by middle and high school students in several African countries. Using this dataset, we conducted two experiments to evaluate the use of large language models (LLM) for grading particularly challenging student answers. In experiment 1, we use a variety of LLM-driven approaches, including zero-shot, few-shot, and chain-of-thought prompting, to grade the 1% of student answers that a rule-based classifier fails to grade accurately. We find that the best-performing approach — chain-of-thought prompting — accurately scored 97% of these edge cases, effectively boosting the overall accuracy of the grading from 96% to 97%. In experiment 2, we aim to better understand the consequential validity of the improved grading accuracy by passing grades generated by the best-performing LLM-based approach to a Bayesian Knowledge Tracing (BKT) model, which estimated student mastery of specific lessons. We find that modest improvements in model accuracy can lead to significant changes in mastery estimation. Where the rule-based classifier misclassified the mastery status of 6.9% of students across completed lessons, using the LLM chain-of-thought approach reduced this to 2.6%. These findings suggest that LLMs could be valuable for grading fill-in questions in mathematics education, potentially enabling wider adoption of open-response questions in learning systems.
- Open Access
- 28 January 2025
- Published by Elsevier in International Journal of Artificial Intelligence in Education
- Vol. 35 (1) , 282-303
Supporting Literacy Assessment in West Africa: Using State-of-the-Art Speech Models to Assess Oral Reading Fluency
- 28 January 2025
- Published by Elsevier in International Journal of Artificial Intelligence in Education
- Vol. 35 (1) , 282-303
Open AccessSupporting Literacy Assessment in West Africa: Using State-of-the-Art Speech Models to Assess Oral Reading Fluency