Self-Prioritizing Multi-Agent Reinforcement Learning for Conflict Resolution in Air Traffic Control with Limited Instructions
Open Access
Abstract
Air traffic control (ATC) relies on a series of complex tasks, the most crucial aspect of which is to assure safe separation between aircraft. Due to the increase in air traffic, decision support systems and safe and robust automation of ATC tasks are of high value. Automated conflict resolution has been an active area of research for decades, and in more recent years, reinforcement learning has been suggested as a powerful alternative to traditional algorithms. Reinforcement learning using discrete action spaces often require large action spaces to cover all combinations of actions, which can make them difficult to train. On the other hand, models with continuous action spaces require much lower dimensionality but often learn to solve conflicts by using a large number of exceedingly small actions. This makes them more suitable for decentralized ATC, such as in unmanned or free-flight airspace. In this paper, we present a novel multi-agent reinforcement learning method with a continuous action space that significantly reduces the number of actions by means of a learning-based priority mechanism. We demonstrate how this can keep the number of actions to a minimum while successfully resolving conflicts with little overhead in the distance required for the aircraft to reach their exit points. As such, the proposed solution is well-suited for centralized ATC, where the number of directives that can be transmitted to aircraft is limited.Keywords
Funding Information
- LFV (TRV 2019/36272)
- Trafikverket (TRV 2019/36272)
This publication has 23 references indexed in Scilit:
- A survey on reinforcement learning in aviation applicationsEngineering Applications of Artificial Intelligence, 2024
- Analysis of the impact of traffic density on training of reinforcement learning based conflict resolution methods for dronesEngineering Applications of Artificial Intelligence, 2024
- General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolutionTransportation Research Part C: Emerging Technologies, 2023
- Physics Informed Deep Reinforcement Learning for Aircraft Conflict ResolutionIEEE Transactions on Intelligent Transportation Systems, 2021
- Deep reinforcement learning based conflict detection and resolution in air traffic controlIET Intelligent Transport Systems, 2019
- Automated Conflict Resolution in Air Traffic ManagementINCAS BULLETIN, 2017
- The Decentralized POMDP Framework2016
- Monte Carlo Optimization for Conflict Resolution in Air Traffic ControlIEEE Transactions on Intelligent Transportation Systems, 2006
- Conflict resolution problems for air traffic management systems solved with mixed integer programmingIEEE Transactions on Intelligent Transportation Systems, 2002
- On optimal cooperative conflict resolution for air traffic management systemsIEEE Transactions on Intelligent Transportation Systems, 2000