Aaron O. Thomas

As tools that produce AI-generated text evolve rapidly and increasingly integrate into our everyday lives, it is essential to understand how they are used and what factors influence people to include AI-generated text in their written work. To directly assess the use of text produced by generative-AI tools and to begin identifying factors that may influence its use, we quantitatively measured, at the individual-word level, the percentage of human- and AI-generated text in undergraduate students’ essays. These results produced a dataset used to identify factors influencing students' use of text generated by generative AI. Our analysis of the percentage of human- and AI-generated text across over 300 student essays revealed that usage of AI-generated text in a student’s essay varied by major, area of study, and a student's academic ranking in college. These data demonstrate the feasibility of tracking students’ use of generative AI in written work, identifying factors influencing how AI-generated content is used, and beginning to provide insights into how students choose to incorporate it into their essays.

Objective. To determine if student self-reported use of optional supplemental material impacts the ability to accurately calibrate performance on a low-stakes assessment. Methods. An instructor created an optional supplemental material in the form of an online quiz activity. Students were asked to report if they used the supplemental material as well as to predict and postdict their performance on an in-class assessment. The relative accuracy of the predictions and postdictions, as well as the assessment grades and overall course grades, were compared between students who reported using the supplemental material and those who reported not using the supplemental material. Results. More than half of the students (60%) reported using the supplemental material. Most students underpredicted their performance, but there was no difference based on supplemental material use or non-use in the predictions (-1.2 vs. -1.0) or the postdictions (-1.3 vs. -1.0). Students who reported using the supplemental material performed better on both the low-stakes assessment (7.7 vs 7.2 out of 10) and overall in the course (87.02% vs. 84.88%). Conclusion. Student self-reported use of optional supplemental material does not appear to affect calibration accuracy.