A multi-aircraft co-operative trajectory planning model under dynamic thunderstorm cells using decentralized deep reinforcement learning

• Main Authors: Pang, Bizhao, Hu, Xinting, Zhang, Mingcheng, Alam, Sameer, Lulli, Guglielmo
• Other Authors: School of Mechanical and Aerospace Engineering
• Format: Article
• Language: English
• Published: 2025
• Subjects: Engineering; Air traffic management; Autonomous trajectory planning; Deep reinforcement learning; Dynamic thunderstorm cells; Climate change
• Online Access: https://hdl.handle.net/10356/182740
• Institution: Nanyang Technological University

Climate change induces an increased frequency of adverse weather, particularly thunderstorms, posing significant safety and efficiency challenges in en route airspace, especially in oceanic regions with limited air traffic control services. These conditions require multi-aircraft cooperative trajectory planning to avoid both dynamic thunderstorms and other aircraft. Existing literature has typically relied on centralized approaches and single-agent principles, which lack coordination and robustness when surrounding aircraft or thunderstorms change paths, leading to scalability issues due to heavy trajectory regeneration needs. To address these gaps, this paper introduces a multi-agent cooperative method for autonomous trajectory planning. The problem is modeled as a Decentralized Markov Decision Process (DEC-MDP) and solved using an Independent Deep Deterministic Policy Gradient (IDDPG) learning framework. A shared actor-critic network is trained using combined experiences from all aircraft to optimize joint behavior. During execution, each aircraft acts independently based on its own observations, with coordination ensured through the shared policy. The model is validated through extensive simulations, including uncertainty analysis, baseline comparisons, and ablation studies. Under known thunderstorm paths, the model achieved a 2 % loss of separation rate, increasing to 4 % with random storm paths. ETA uncertainty analysis demonstrated the model's robustness, while baseline comparisons with the Fast Marching Tree and centralized DDPG highlighted its scalability and efficiency. These findings contribute to advancing autonomous aircraft operations.