Tracy Arner
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Tracy Arner has a total of 42 co-authors. Here are the top co-authors by number of shared publications:
- Danielle S. McNamara
Learning Engineering Institute, Arizona State University, Tempe, AZ 85281, USA
17 shared publications - Renu Balyan
SUNY Old Westbury, Long Island, NY, USA
6 shared publications - Jiachen Gong
Learning Engineering Institute, Arizona State University, Tempe, AZ, USA
4 shared publications - Micah Watanabe
Learning Engineering Institute, Arizona State University, 1151 S Forest Ave, Tempe, AZ 85287, USA
4 shared publications - Katerina Christhilf
Arizona State University
3 shared publications - Linh Huynh
Learning Engineering Institute, Arizona State University, 120 Cady Mall, Tempe, AZ 85281, USA
3 shared publications
Latest Publications
- 28 June 2026
- Published by Springer Nature
- p. 315-324
Re-imagine Knowledge Tracing with Student Agency in a Generative AI Language Tutor
- 28 June 2026
- Published by Springer Nature
- p. 315-324
Re-imagine Knowledge Tracing with Student Agency in a Generative AI Language Tutor
- Open Access
- 13 March 2026
- Published by MDPI AG in Electronics
- Vol. 15 (6) , 1209
Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
- 13 March 2026
- Published by MDPI AG in Electronics
- Vol. 15 (6) , 1209
Open AccessCognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
- Open Access
- 12 February 2026
- Published by MDPI AG in Education Sciences
- Vol. 16 (2) , 300
Characterizing User Needs for GenAI Incorporation in Educational Games
This work explores user needs for educational games and gamification that incorporates Generative Artificial Intelligence (GenAI). As GenAI is increasingly incorporated in educational settings, we must consider both the wide-spanning literature on gamification and games that have been shown to benefit learning, and characterize the needs and desires of relevant stakeholders in developing educational games that incorporate GenAI generally, and specifically for higher education. A mixed-methods questionnaire inquired 345 undergraduate students about their perceptions, use patterns, needs, and desires related to GenAI, educational and non-educational games, and text-based games. GenAI tools are widely used for educational purposes already, but mostly as a supplementary source. Despite the wide use, participants expressed being concerned with accuracy, transparency, and quality. Participants also expressed a desire for an educational game/tool to have scaffolded interactions and to help with learning material in math, science, and language arts. Taken together the findings provide a road map and specific recommendations for developing an educational game incorporating GenAI. The roadmap includes instructional design (i.e., the gamified tools’ content and type(s) of instruction and interaction) through information regarding preferred platforms, game genres, gamified properties (e.g., characters, challenges), and lastly, clear information about concerns students have related to trust and equity that will need to be addressed in an educational game incorporating GenAI.
- 12 February 2026
- Published by MDPI AG in Education Sciences
- Vol. 16 (2) , 300
Open AccessCharacterizing User Needs for GenAI Incorporation in Educational Games
This work explores user needs for educational games and gamification that incorporates Generative Artificial Intelligence (GenAI). As GenAI is increasingly incorporated in educational settings, we must consider both the wide-spanning literature on gamification and games that have been shown to benefit learning, and characterize the needs and desires of relevant stakeholders in developing educational games that incorporate GenAI generally, and specifically for higher education. A mixed-methods questionnaire inquired 345 undergraduate students about their perceptions, use patterns, needs, and desires related to GenAI, educational and non-educational games, and text-based games. GenAI tools are widely used for educational purposes already, but mostly as a supplementary source. Despite the wide use, participants expressed being concerned with accuracy, transparency, and quality. Participants also expressed a desire for an educational game/tool to have scaffolded interactions and to help with learning material in math, science, and language arts. Taken together the findings provide a road map and specific recommendations for developing an educational game incorporating GenAI. The roadmap includes instructional design (i.e., the gamified tools’ content and type(s) of instruction and interaction) through information regarding preferred platforms, game genres, gamified properties (e.g., characters, challenges), and lastly, clear information about concerns students have related to trust and equity that will need to be addressed in an educational game incorporating GenAI.
- Open Access
Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
Open AccessCognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
- Open Access
- 3 February 2026
- Published by MDPI AG in Preprints.org
Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
- 3 February 2026
- Published by MDPI AG in Preprints.org
Open AccessCognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models
Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.
- Open Access
- 19 December 2025
- Published by MDPI AG in Education Sciences
- Vol. 16 (1) , 6
Integrating Reading, Writing, and Digital Tools in Science: A Participatory-Design Study of the InSPECT Framework
The purpose of this study was to engage high school science teachers as co-design partners in refining and extending instructional frameworks to support multiple-document reading and writing in science classrooms. Using a participatory mixed-methods design, the project adapted the InSPECT framework for secondary science, developed professional development (PD) materials to introduce the framework, and explored the role of generative artificial intelligence (AI) in lesson planning. Five virtual focus group sessions guided the co-design of PD activities, followed by a pilot implementation in one biology classroom. Data included focus group and interview transcripts, surveys, and student work artifacts. Analyses examined teachers’ perceptions of PD features, framework usability, and student engagement. Teachers valued PD that was practical, relevant, and feasible within classroom constraints and described the frameworks as clear, stepwise structures that supported lesson design and literacy integration. Student work showed that paraphrasing was an accessible entry point, while bridging, elaboration, and source evaluation required additional modeling. Teachers viewed generative AI as a promising planning aid but expressed concerns about accuracy and ethics. Findings informed revisions emphasizing discipline-specific exemplars, scaffolds for higher-order strategies, and AI-literacy modules, illustrating how participatory design can yield feasible, teacher-centered PD.
- 19 December 2025
- Published by MDPI AG in Education Sciences
- Vol. 16 (1) , 6
Open AccessIntegrating Reading, Writing, and Digital Tools in Science: A Participatory-Design Study of the InSPECT Framework
The purpose of this study was to engage high school science teachers as co-design partners in refining and extending instructional frameworks to support multiple-document reading and writing in science classrooms. Using a participatory mixed-methods design, the project adapted the InSPECT framework for secondary science, developed professional development (PD) materials to introduce the framework, and explored the role of generative artificial intelligence (AI) in lesson planning. Five virtual focus group sessions guided the co-design of PD activities, followed by a pilot implementation in one biology classroom. Data included focus group and interview transcripts, surveys, and student work artifacts. Analyses examined teachers’ perceptions of PD features, framework usability, and student engagement. Teachers valued PD that was practical, relevant, and feasible within classroom constraints and described the frameworks as clear, stepwise structures that supported lesson design and literacy integration. Student work showed that paraphrasing was an accessible entry point, while bridging, elaboration, and source evaluation required additional modeling. Teachers viewed generative AI as a promising planning aid but expressed concerns about accuracy and ethics. Findings informed revisions emphasizing discipline-specific exemplars, scaffolds for higher-order strategies, and AI-literacy modules, illustrating how participatory design can yield feasible, teacher-centered PD.
- Open Access
- 17 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 381-386
L2 English and Culture as Factors in College Math Achievement
- 17 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 381-386
Open AccessL2 English and Culture as Factors in College Math Achievement
- Open Access
- 4 June 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (11) , 6316
Awareness to Action: Student Knowledge of and Responses to an Early Alert System
Introduction: Student retention is a critical issue in higher education. Universities have responded by implementing supports like early alert systems. Objective: We investigated students’ knowledge of and experiences with an early alert system designed to enhance academic persistence. Method: We surveyed (N = 356) undergraduates at a large public university in the U.S. The survey was researcher-created and administered online. Participants self-selected into the study and provided informed consent prior to beginning the study. Data were coded by experts, who achieved excellent IRR. The analyses were frequency-based to understand diverse student knowledge, experiences, and responses with early alert systems. Results: Students commonly reported experiencing negative emotions after receiving an alert, but also reported that receiving an alert motivated them to increase their course attendance, improve their study habits, and participate more in class. Finally, students indicated that receiving an early alert facilitated supportive interactions with instructors, though student communication with academic advisors was less frequent. Student recommendations for system improvement included using positive language in alerts and providing actionable guidance. Conclusions: These results provide new insight into student views of early alert systems and suggest that these systems can positively impact students in need of support.
- 4 June 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (11) , 6316
Open AccessAwareness to Action: Student Knowledge of and Responses to an Early Alert System
Introduction: Student retention is a critical issue in higher education. Universities have responded by implementing supports like early alert systems. Objective: We investigated students’ knowledge of and experiences with an early alert system designed to enhance academic persistence. Method: We surveyed (N = 356) undergraduates at a large public university in the U.S. The survey was researcher-created and administered online. Participants self-selected into the study and provided informed consent prior to beginning the study. Data were coded by experts, who achieved excellent IRR. The analyses were frequency-based to understand diverse student knowledge, experiences, and responses with early alert systems. Results: Students commonly reported experiencing negative emotions after receiving an alert, but also reported that receiving an alert motivated them to increase their course attendance, improve their study habits, and participate more in class. Finally, students indicated that receiving an early alert facilitated supportive interactions with instructors, though student communication with academic advisors was less frequent. Student recommendations for system improvement included using positive language in alerts and providing actionable guidance. Conclusions: These results provide new insight into student views of early alert systems and suggest that these systems can positively impact students in need of support.
- 24 March 2025
- Published by Wiley in Journal of Computer Assisted Learning
- Vol. 41 (2)
Choose Your Own Adventure: Interactive E‐Books to Improve Word Knowledge and Comprehension Skills
Background: The affordances of technology, such as e‐books, offer the opportunity to increase engagement and provide personalised feedback to promote students' learning outcomes. E‐books that encourage the use of comprehension monitoring strategies in real time may support stronger outcomes.Objectives: The purpose of this feasibility study was to examine the potential impact of reading digital interactive e‐books, Word Knowledge E‐Books (WKe‐Books), that taught word‐meaning and deeper text‐meaning strategies to support reading comprehension with third‐fifth grade students.Methods: Students (N = 435) read two WKe‐Books, that taught word learning and comprehension monitoring strategies in the service of learning vocabulary and targeted science concepts about hurricanes. We investigated whether specific comprehension strategies–(1) word learning and strategies that supported general reading comprehension, (2) summarisation, and (3) question generation–show promise of effectiveness in building vocabulary knowledge and comprehension skills in the WKe‐Books. Students were assigned to read one of three versions of each WKe‐Book. The books employed a choose‐your‐adventure format with embedded comprehension questions that provided students with immediate feedback.Results and Conclusions: Students demonstrated significant gains in word learning and the targeted hurricane concepts. Hierarchical Linear Modelling (HLM) revealed that no one strategy was associated with larger gains. Performance on the embedded questions in the books was associated with greater posttest outcomes. These findings suggest that the affordances offered in the WKe‐Books can effectively support students' development of reading‐related skills, including strategy use. Further, this work discusses implications for the future development of e‐books that can enhance engagement and improve reading comprehension.
- 24 March 2025
- Published by Wiley in Journal of Computer Assisted Learning
- Vol. 41 (2)
Choose Your Own Adventure: Interactive E‐Books to Improve Word Knowledge and Comprehension Skills
Background: The affordances of technology, such as e‐books, offer the opportunity to increase engagement and provide personalised feedback to promote students' learning outcomes. E‐books that encourage the use of comprehension monitoring strategies in real time may support stronger outcomes.Objectives: The purpose of this feasibility study was to examine the potential impact of reading digital interactive e‐books, Word Knowledge E‐Books (WKe‐Books), that taught word‐meaning and deeper text‐meaning strategies to support reading comprehension with third‐fifth grade students.Methods: Students (N = 435) read two WKe‐Books, that taught word learning and comprehension monitoring strategies in the service of learning vocabulary and targeted science concepts about hurricanes. We investigated whether specific comprehension strategies–(1) word learning and strategies that supported general reading comprehension, (2) summarisation, and (3) question generation–show promise of effectiveness in building vocabulary knowledge and comprehension skills in the WKe‐Books. Students were assigned to read one of three versions of each WKe‐Book. The books employed a choose‐your‐adventure format with embedded comprehension questions that provided students with immediate feedback.Results and Conclusions: Students demonstrated significant gains in word learning and the targeted hurricane concepts. Hierarchical Linear Modelling (HLM) revealed that no one strategy was associated with larger gains. Performance on the embedded questions in the books was associated with greater posttest outcomes. These findings suggest that the affordances offered in the WKe‐Books can effectively support students' development of reading‐related skills, including strategy use. Further, this work discusses implications for the future development of e‐books that can enhance engagement and improve reading comprehension.
- 1 December 2024
- Published by American Psychological Association (APA) in Journal of Applied Research in Memory and Cognition
- Vol. 13 (4) , 465-484
Expert thinking with generative chatbots.
- 1 December 2024
- Published by American Psychological Association (APA) in Journal of Applied Research in Memory and Cognition
- Vol. 13 (4) , 465-484
Expert thinking with generative chatbots.