AutoTA: A Dynamic Intent-Based Virtual Teaching Assistant for Students Using Open Source LLMs
Open Access
- 4 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 118122-118134
Abstract
Large Language Models (LLMs) are explored for their potential to transform education by serving as virtual teaching assistants, offering personalized support through human-like responses to tasks such as content-related questions and coursework guidance. In this study, we present a novel framework that leverages intent classification to enhance the effectiveness of LLMs in this role. Our framework, AutoTA, categorizes student queries into distinct topics— lecture discussions, homework assistance, and syllabus questions—triggering specific conversation chains tailored to each intent. Additionally, we incorporate a custom vector-space filter that refines responses based on filename tracking after intent identification. To evaluate the framework, we used course materials from the undergraduate-level CS course, Computer Incident Response, and compared the performance of several open-source LLMs, including Llama 3.1. Our results show that the framework accurately classifies intent and provides appropriate guidance, measured through quantitative and qualitative metrics. These findings highlight the potential of the proposed framework to enhance personalized learning and improve student engagement. While tested in a computer science course, the framework incorporates diverse assessment types that suggest potential for broader application.Keywords
Funding Information
- Defense Advanced Research Projects Agency (AI-CRAFT AWD16069)
- Visiting Research Assistant in Gyawali’s Laboratory
This publication has 17 references indexed in Scilit:
- Comparing AI in Online Learning: The Transition and Trade-offs Between Intent-Based Learning Assistants and LLM-Chatbots in MOOCs2024
- Faculty Perspectives on the Potential of RAG in Computer Science Higher Education2024
- Towards effective teaching assistants: From intent-based chatbots to LLM-powered teaching assistantsNatural Language Processing Journal, 2024
- CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs2024
- CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes2023
- Large language models in medicineNature Medicine, 2023
- Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming2023
- A Systematic Review of Research on Personalized Learning: Personalized by Whom, to What, How, and for What Purpose(s)?Educational Psychology Review, 2021
- Personalized Attention @ Scale2015
- Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and TheoryHealth Services Research, 2007