An Integrated RAG and Agent-Based Architecture for Automated Assessment in Moodle
Open Access
- 12 May 2026
- journal article
- research article
- Published by MDPI AG in Machine Learning and Knowledge Extraction
- Vol. 8 (5) , 127
Abstract
The growing adoption of Generative AI in education has created opportunities to automate complex pedagogical tasks, yet reliably and scalably assessing open-ended responses remains a challenge. This study proposes and evaluates an architectural solution for integrating a Large Language Model (LLM) into Moodle, combining Retrieval-Augmented Generation (RAG) and AI agent mechanisms to enable automated grading of open-ended student responses. A Moodle instance was deployed for experimental purposes, with 32 students across Bulgarian- and English-language sections, yielding data at the student (N = 32) and task (N = 160) levels, including AI-generated and instructor-assigned scores and system processing logs. The results demonstrate that the proposed system achieves substantial reductions in grading time while maintaining high agreement with expert assessments. Bias analysis revealed minimal systematic deviation across both language groups, indicating that the system preserves assessment objectivity without consistent over- or underestimation based on language. These findings suggest that a combined RAG and agentic LLM architecture can deliver efficient, accurate, and linguistically robust automated assessment within an LMS environment, offering practical design guidelines applicable to other educational platforms and similar systems.Keywords
Funding Information
- European Regional Development Fund (BG-RRP-2.004-0005)
This publication has 35 references indexed in Scilit:
- Automated Grading of Open-Ended Questions in Higher Education Using GenAI ModelsInternational Journal of Artificial Intelligence in Education, 2025
- Explainable educational assistant integrated in Moodle: automated semantic assessment and adaptive tutoring based on NLP and XAIDiscover Artificial Intelligence, 2025
- Developing high-quality, practical, and ethical automated L2 speaking assessmentsSystem, 2025
- A survey of automated essay scoring: Challenges, advances, and futureNeurocomputing, 2025
- Exercise-Aware higher-order Thinking skills Assessment via fine-tuned large language modelKnowledge-Based Systems, 2025
- LLM-generated tips rival expert-created tips in helping students answer quantum-computing questionsEPJ Quantum Technology, 2025
- Harnessing LLMs for multi-dimensional writing assessment: Reliability and alignment with human judgmentsHeliyon, 2024
- Replicating a High-Impact Scientific Publication Using Systems of Large Language ModelsbioRxiv, 2024
- Applying large language models and chain-of-thought for automatic scoringComputers and Education: Artificial Intelligence, 2024
- Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students’ text revision, motivation, and positive emotionsComputers and Education: Artificial Intelligence, 2023