Investigating the Use of Intelligent Tutors Based on Large Language Models: Automated generation of Business Process Management questions using the Revised Bloom's Taxonomy
Open Access
- 4 November 2024
- conference paper
- Published by Sociedade Brasileira de Computação
- p. 1587-1601
Abstract
The construction of assessment artifacts is a complex task, since generating appropriate assessments manually requires in-depth knowledge of both the area to be assessed and the cognitive processes involved in learning. The use of Large Language Models (LLMs) as the basis for the operation of Intelligent Tutoring Systems can assist in this task. This work experiments with the GPT-3.5-Turbo and LLama-2 LLMs as a source of automatic generation of assessment questions. The experiment was carried out using Prompt Engineering techniques to generate questions for the Business Process Management (BPM) discipline. From the experiment, it was possible to observe that both models are capable of generating questions appropriate to the BPM context. It was also identified that, when it received the context and the model of the question to be generated, the LLama-2 model produced questions more appropriate to the desired cognitive level, while the GPT-3.5-Turbo model received only the context and produced a similar response.Keywords
This publication has 21 references indexed in Scilit:
- Defending Language Models Against Image-Based Prompt Attacks via User-Provided Specifications2024
- Opinerium: Subjective Question Generation Using Large Language ModelsIEEE Access, 2024
- Student Performance Prediction Approach Based on Educational Data MiningIEEE Access, 2023
- Few-shot is enough: exploring ChatGPT prompt engineering method for automatic question generation in english educationEducation and Information Technologies, 2023
- BloomBERT: A Deep Learning-Based Cognitive Complexity Classifier of Assessment Questions2023
- Conversational Intelligent Tutoring Systems for Online Learning: What do Students and Tutors Say?2022
- Teaching Business Process Management with a Flipped-Classroom and Problem-Based Learning Approach with the Use of Apromore and Other BPM Software in Graduate Information Systems Courses2021
- Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methodsInteractive Learning Environments, 2018
- Assessment approaches in massive open online courses: Possibilities, challenges and future directionsInternational Review of Education, 2018
- Fundamentals of Business Process Management2018