Hunter McNichols
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hunter McNichols has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Andrew Lan
University of Massachusetts Amherst
3 shared publications - Fareya Ikram
University of Massachusetts Amherst
2 shared publications - Candace Walkington
Department of Teaching and Learning, Southern Methodist University, Dallas, USA
1 shared publication - Junyang Lu
University of Massachusetts Amherst
1 shared publication Neil Heffernan
Worcester Polytechnic Institute
1 shared publication- Nischal Ashok Kumar
University of Massachusetts Amherst
1 shared publication
Latest Publications
- 27 June 2026
- Published by Springer Nature
- p. 233-247
A Multi-agent Approach to Validate and Refine LLM-Generated Personalized Math Problems
- 27 June 2026
- Published by Springer Nature
- p. 233-247
A Multi-agent Approach to Validate and Refine LLM-Generated Personalized Math Problems
- Top Cited PapersOpen Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 53-63
The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 53-63
Top Cited PapersOpen AccessThe StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course
- 21 July 2025
- Published by Online Science Publishing in American Journal of Education and Learning
- Vol. 10 (2) , 178-189
Using item features to calibrate educational test items: Comparing artificial intelligence and classical approaches
Educational test items are typically calibrated onto a score scale using item response theory (IRT). This approach requires administering the items to hundreds of test takers to characterize their difficulty. For educational tests designed for criterion-referenced purposes, characterizing item difficulty in this way presents two problems: one theoretical, the other practical. Theoretically, tests designed to provide criterion-referenced information should report test takers’ performance with respect to the knowledge and skills they have mastered, rather than on how well they performed relative to others. The traditional IRT calibration approach expresses item difficulty on a scale determined solely by test takers’ performance on the items. Practically, the traditional IRT approach requires large numbers of test takers, who are not always available and who are not always motivated to do well. In this study, we use the construct-relevant features of test items to characterize their difficulty. In one approach, we code the item features; two other approaches are based on artificial intelligence (chain-of-thought prompting and LLM finetuning). The results indicate the coding and LLM finetuning approaches reflect the difficulty parameters calibrated using IRT, accounting for approximately 60% of the variation. These results suggest educational test items can be calibrated using construct-relevant features of the items, rather than only administering them to samples of test takers. Implications for future research and practice in this area are discussed.
- 21 July 2025
- Published by Online Science Publishing in American Journal of Education and Learning
- Vol. 10 (2) , 178-189
Using item features to calibrate educational test items: Comparing artificial intelligence and classical approaches
Educational test items are typically calibrated onto a score scale using item response theory (IRT). This approach requires administering the items to hundreds of test takers to characterize their difficulty. For educational tests designed for criterion-referenced purposes, characterizing item difficulty in this way presents two problems: one theoretical, the other practical. Theoretically, tests designed to provide criterion-referenced information should report test takers’ performance with respect to the knowledge and skills they have mastered, rather than on how well they performed relative to others. The traditional IRT calibration approach expresses item difficulty on a scale determined solely by test takers’ performance on the items. Practically, the traditional IRT approach requires large numbers of test takers, who are not always available and who are not always motivated to do well. In this study, we use the construct-relevant features of test items to characterize their difficulty. In one approach, we code the item features; two other approaches are based on artificial intelligence (chain-of-thought prompting and LLM finetuning). The results indicate the coding and LLM finetuning approaches reflect the difficulty parameters calibrated using IRT, accounting for approximately 60% of the variation. These results suggest educational test items can be calibrated using construct-relevant features of the items, rather than only administering them to samples of test takers. Implications for future research and practice in this area are discussed.