Collaborative multiagent reinforcement learning schemes for air traffic management
Authors:
- 1 July 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
In this work we investigate the use of hierarchical collaborative reinforcement learning methods (H-CMARL) for the computation of joint policies to resolve congestion problems in the Air Traffic Management (ATM) domain. In particular, to address cases where the demand of airspace use exceeds capacity, we consider agents representing flights, who need to decide jointly on ground delays at the pre-tactical stage of operations, towards executing their trajectories while adhering to airspace capacity constraints. In doing so, agents collaborate, applying collaborative multi-agent reinforcement learning methods. Specifically, starting from a multiagent Markov Decision Process problem formulation, we introduce a flat and a hierarchical collaborative multiagent reinforcement learning method at two levels (the ground and an abstract one). To quantitatively assess the quality of solutions of the proposed approaches and show the potential of the hierarchical method in resolving the demand-capacity balance problems, we provide experimental results on real-world evaluation cases, where we measure the average delay of flights and the number of flights with delays.Keywords
This publication has 12 references indexed in Scilit:
- Learning Policies for Resolving Demand-Capacity Imbalances During Pre-tactical Air Traffic ManagementLecture Notes in Computer Science, 2017
- A neural model of hierarchical reinforcement learningPLOS ONE, 2017
- Visual Analytics of Movement2013
- Congestion games with failuresDiscrete Applied Mathematics, 2011
- A multiagent approach to managing air traffic flowAutonomous Agents and Multi-Agent Systems, 2010
- Social optimality and cooperation in nonatomic congestion gamesJournal of Economic Theory, 2004
- Optimization problems in congestion control2002
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999
- Agents in Traffic Modelling — From Reactive to Social Behaviour1999
- A class of games possessing pure-strategy Nash equilibriaInternational Journal of Game Theory, 1973