A multi-agent reinforcement learning approach for flight speed control systems

• Main Author: Kanupriya, Malhotra
• Other Authors: Sameer Alam
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Aeronautical engineering::Air navigation; Engineering::Computer science and engineering::Computing methodologies::Simulation and modeling
• Online Access: https://hdl.handle.net/10356/159164
• Institution: Nanyang Technological University

With increasing air traffic, there is an ever-growing need for Air Traffic Controllers (ATCO) to efficiently manage air traffic and congestion. Congestion often leads to increase in delays in the Terminal Maneuvering Area (TMA), which is one of the primary challenges that is being faced by ATCO. Introduction of approaches such as the Extended Arrival Manager (E-AMAN) propose solutions whereby flights are scheduled early, much before they enter the TMA, to absorb flight delays in the TMA that could potentially cause large amounts of fuel burn and have detrimental environmental impacts. However, for practical implications of such an approach, a speed management system is required, whereby, flights can coordinate to effectively absorb system-level delays in their en-route phase. This report proposes a Multi-Agent System (MAS) approach using Deep Reinforcement Learning to model and train flights as agents which can coordinate with each other to effectively absorb system-level delays. The simulations use MultiAgent POsthumous Credit Assignment (MA-POCA) in Unity to model flights as agents that are part of a cooperative MAS. Two reward approaches have been compared, where flights are either provided with complete penalty for exceeding the required delay, or partial penalty for exceeding the required delay. Findings highlight the percentage of maximum delay absorbed by each flight, along with the distribution of flight delay absorption in the flight plan. Initial testings reveal an average of 3.3 minutes of system-level delay absorptions from a required 4 minutes of delay.