Evaluation of Multi-Model Architecture Against Single-Model PPO in the MiniGrid Environment
Open Access
- 19 July 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (14) , 3170
Abstract
Reward sparsity is a considerable problem in reinforcement learning when solving complex problems. In an environment where the reward is given only after successfully solving a problem in multiple steps, the signals for optimization can be too weak, preventing the agent from learning. However, this can be solved by breaking the problem into an array of simpler tasks that the agent can learn to perform. This paper presents a possible solution that uses a mixture-of-experts architecture. Separate models are used for planning, routing expert outputs, and solving singular sub-tasks. The solution was implemented in the MiniGrid environment, which allows for the design of arbitrarily complex problems. The results presented in this paper show that a multi-model agent can solve more complex problems comprising multiple steps than a single reinforcement learning agent.Keywords
This publication has 25 references indexed in Scilit:
- Benefits of Using LLM for Long-Term Planning with Distilled Subtask Model Compared to End-to-End Reinforcement Learning in the MiniGrid SimulatorElectronics, 2026
- Multi-Agent Collaboration for 3D Human Pose Estimation and Its Potential in Passenger-Gathering Behavior Early WarningElectronics, 2025
- Exploring the Cognitive Capabilities of Large Language Models in Autonomous and Swarm Navigation SystemsElectronics, 2025
- MoE-World: A Mixture-of-Experts Architecture for Multi-Task World ModelsElectronics, 2025
- Large Language Models for Structured Task Decomposition in Reinforcement Learning Problems with Sparse RewardsMachine Learning and Knowledge Extraction, 2025
- Optimizing Agent Behavior in the MiniGrid Environment Using Reinforcement Learning Based on Large Language ModelsApplied Sciences, 2025
- Trust Region Policy OptimizationarXiv, 2017
- Playing Atari with Deep Reinforcement LearningarXiv, 2013
- Twenty Years of Mixture of ExpertsIEEE Transactions on Neural Networks and Learning Systems, 2012
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991