Bridging LMS and Generative AI: Dynamic Course Content Integration (DCCI) for Connecting LLMs to Course Content -- The Ask ME Assistant
Authors:
Preprint
Abstract
The integration of Large Language Models (LLMs) with Learning Management Systems (LMSs) has the potential to enhance task automation and accessibility in education. However, hallucination where LLMs generate inaccurate or misleading information remains a significant challenge. This study introduces the Dynamic Course Content Integration (DCCI) mechanism, which dynamically retrieves and integrates course content and curriculum from Canvas LMS into the LLM-powered assistant, Ask ME. By employing prompt engineering to structure retrieved content within the LLM's context window, DCCI ensures accuracy, relevance, and contextual alignment, mitigating hallucination. To evaluate DCCI's effectiveness, Ask ME's usability, and broader student perceptions of AI in education, a mixed-methods approach was employed, incorporating user satisfaction ratings and a structured survey. Results from a pilot study indicate high user satisfaction (4.614/5), with students recognizing Ask ME's ability to provide timely and contextually relevant responses for both administrative and course-related inquiries. Additionally, a majority of students agreed that Ask ME's integration with course content in Canvas LMS reduced platform-switching, improving usability, engagement, and comprehension. AI's role in reducing classroom hesitation and fostering self-directed learning and intellectual curiosity was also highlighted. Despite these benefits and positive perception of AI tools, concerns emerged regarding over-reliance on AI, accuracy limitations, and ethical issues such as plagiarism and reduced student-teacher interaction. These findings emphasize the need for strategic AI implementation, ethical safeguards, and a pedagogical framework that prioritizes human-AI collaboration over substitution.Keywords
All Related Versions
This publication has 69 references indexed in Scilit:
- Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language modelsArtificial Intelligence Review, 2024
- The rapid rise of generative AI and its implications for academic integrity: Students’ perceptions and use of chatbots for assistance with assessmentsComputers and Education: Artificial Intelligence, 2024
- Special considerations for the use of AI tools by PEERs as a learning and communication aidJournal of Cellular Physiology, 2024
- Artificial Intelligence in Education: Implications for Policymakers, Researchers, and PractitionersTechnology, Knowledge and Learning, 2024
- The impact of a chatbot working as an assistant in a course for supporting student learning and engagementComputer Applications in Engineering Education, 2024
- AI literacy and its implications for prompt engineering strategiesComputers and Education: Artificial Intelligence, 2024
- Academic Surgery in the Era of Large Language ModelsJAMA Surgery, 2024
- Formative Feedback on Student-Authored Summaries in Intelligent Textbooks Using Large Language ModelsInternational Journal of Artificial Intelligence in Education, 2024
- Artificial intelligence (AI) learning tools in K-12 education: A scoping reviewJournal of Computers in Education, 2024
- Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students’ text revision, motivation, and positive emotionsComputers and Education: Artificial Intelligence, 2023