Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMs
Preprint
Abstract
There has been a growing interest in developing learner models to enhance learning and teaching experiences in educational environments. However, existing works have primarily focused on structured environments relying on meticulously crafted representations of tasks, thereby limiting the agent's ability to generalize skills across tasks. In this paper, we aim to enhance the generalization capabilities of agents in open-ended text-based learning environments by integrating Reinforcement Learning (RL) with Large Language Models (LLMs). We investigate three types of agents: (i) RL-based agents that utilize natural language for state and action representations to find the best interaction strategy, (ii) LLM-based agents that leverage the model's general knowledge and reasoning through prompting, and (iii) hybrid LLM-assisted RL agents that combine these two strategies to improve agents' performance and generalization. To support the development and evaluation of these agents, we introduce PharmaSimText, a novel benchmark derived from the PharmaSim virtual pharmacy environment designed for practicing diagnostic conversations. Our results show that RL-based agents excel in task completion but lack in asking quality diagnostic questions. In contrast, LLM-based agents perform better in asking diagnostic questions but fall short of completing the task. Finally, hybrid LLM-assisted RL agents enable us to overcome these limitations, highlighting the potential of combining RL and LLMs to develop high-performing agents for open-ended learning environments.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Assessing student errors in experimentation using artificial intelligence and large language models: A comparative study with human ratersComputers and Education: Artificial Intelligence, 2023
- Simulated Learners in Educational Technology: A Systematic Literature Review and a Turing-like TestInternational Journal of Artificial Intelligence in Education, 2023
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- Approximately Optimal Teaching of Approximately Optimal LearnersIEEE Transactions on Learning Technologies, 2017
- Faster Teaching via POMDP PlanningCognitive Science, 2015
- A Teachable Agent Game Engaging Primary School Children to Learn Arithmetic Concepts and ReasoningInternational Journal of Artificial Intelligence in Education, 2014