Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems
Preprint
Abstract
Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem, identifying their best-response policies. Solving this bi-level optimization problem is computationally intensive, as it requires repeatedly solving multi-agent reinforcement learning problems under various compositions for both types of agents. To address these challenges, we propose a novel hypernetwork-based framework that jointly optimizes the system's composition and agent policies. Unlike traditional methods that train separate policy networks for each composition, the proposed framework generates policies for both controllable and uncontrollable agents through a unified hypernetwork. This approach enables efficient information sharing across similar configurations, thereby reducing computational overhead. Additional improvements are achieved by incorporating reward parameter optimization and mean action networks. Using real-world New York City taxi data, we demonstrate that our framework outperforms existing methods in approximating equilibrium policies. Our experimental results show significant improvements in key performance metrics, such as order response rate and served demand, highlighting the practical utility of controlling agents and their potential to enhance decision-making in PCMAS.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- A brief review of hypernetworks in deep learningArtificial Intelligence Review, 2024
- Beyond centralization: Non-cooperative perimeter control with extended mean-field reinforcement learning in urban road networksTransportation Research Part B: Methodological, 2024
- Two-Sided Deep Reinforcement Learning for Dynamic Mobility-on-Demand Management with Mixed AutonomyTransportation Science, 2023
- The relationship between hospital ownership, in-hospital mortality, and medical expenses: an analysis of three common conditions in ChinaArchives of Public Health, 2023
- Learning how to dynamically route autonomous vehicles on shared roadsTransportation Research Part C: Emerging Technologies, 2021
- A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to AI-guided driving policy learningTransportation Research Part C: Emerging Technologies, 2021
- Reward design for driver repositioning using multi-agent reinforcement learningTransportation Research Part C: Emerging Technologies, 2020
- Algorithmic Mechanism DesignGames and Economic Behavior, 2001