Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty
Recently, the advances in reinforcement learning have enabled an artificial intelligent agent to solve many challenging problems (e.g. AlphaGo) at unprecedented levels. However, the robustness of reinforcement learning in safety critical operation remains unclear. In this work, the applicability of...
Saved in:
Main Authors: | , , , , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2020
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/144709 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-144709 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1447092020-11-21T20:10:38Z Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty Pham, Duc-Thinh Tran, Ngoc Phu Goh, Sim Kuan Alam, Sameer Duong, Vu 2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF) Air Traffic Management Research Institute Engineering::Aeronautical engineering Air Traffic Command and Control Reinforcement Learning Recently, the advances in reinforcement learning have enabled an artificial intelligent agent to solve many challenging problems (e.g. AlphaGo) at unprecedented levels. However, the robustness of reinforcement learning in safety critical operation remains unclear. In this work, the applicability of reinforcement learning in Air Traffic Control was explored. We focus on building an algorithm to automate flight conflict resolution which is the ultimate goal of air traffic control. For that purpose, a simulator, that provides a learning environment for reinforcement learning, was developed to simulate a variety of air traffic scenarios. We propose a variant of the reinforcement learning approach to resolve conflict in airspace and investigate the performance of the method in achieving that. A reinforcement learning model, specifically a deep deterministic policy gradient, was adopted to learn the conflict resolution with continuous action spaces. Experimental results demonstrate that our proposed method is effective in resolving the conflict between two aircraft even in the presence of uncertainty. The accuracy of our model is 87% at different uncertainty levels. Our findings suggest that reinforcement learning is a promising approach to conflict resolution. Civil Aviation Authority of Singapore (CAAS) Accepted version This research has been partially supported under Air Traffic Management Research Institute (NTU-CAAS) Grant No. M4062429.052 2020-11-20T03:00:16Z 2020-11-20T03:00:16Z 2019 Conference Paper Pham, D.-T., Tran, N. P., Goh, S. K., Alam, S., & Duong, V. (2019). Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty. Proceedings of the 2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF), 1-6. doi:10.1109/RIVF.2019.8713624 978-1-5386-9313-1 https://hdl.handle.net/10356/144709 10.1109/RIVF.2019.8713624 2-s2.0-85066629686 1 6 en © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work is available at: https://doi.org/10.1109/RIVF.2019.8713624 application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Aeronautical engineering Air Traffic Command and Control Reinforcement Learning |
spellingShingle |
Engineering::Aeronautical engineering Air Traffic Command and Control Reinforcement Learning Pham, Duc-Thinh Tran, Ngoc Phu Goh, Sim Kuan Alam, Sameer Duong, Vu Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
description |
Recently, the advances in reinforcement learning have enabled an artificial intelligent agent to solve many challenging problems (e.g. AlphaGo) at unprecedented levels. However, the robustness of reinforcement learning in safety critical operation remains unclear. In this work, the applicability of reinforcement learning in Air Traffic Control was explored. We focus on building an algorithm to automate flight conflict resolution which is the ultimate goal of air traffic control. For that purpose, a simulator, that provides a learning environment for reinforcement learning, was developed to simulate a variety of air traffic scenarios. We propose a variant of the reinforcement learning approach to resolve conflict in airspace and investigate the performance of the method in achieving that. A reinforcement learning model, specifically a deep deterministic policy gradient, was adopted to learn the conflict resolution with continuous action spaces. Experimental results demonstrate that our proposed method is effective in resolving the conflict between two aircraft even in the presence of uncertainty. The accuracy of our model is 87% at different uncertainty levels. Our findings suggest that reinforcement learning is a promising approach to conflict resolution. |
author2 |
2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF) |
author_facet |
2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF) Pham, Duc-Thinh Tran, Ngoc Phu Goh, Sim Kuan Alam, Sameer Duong, Vu |
format |
Conference or Workshop Item |
author |
Pham, Duc-Thinh Tran, Ngoc Phu Goh, Sim Kuan Alam, Sameer Duong, Vu |
author_sort |
Pham, Duc-Thinh |
title |
Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
title_short |
Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
title_full |
Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
title_fullStr |
Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
title_full_unstemmed |
Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
title_sort |
reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty |
publishDate |
2020 |
url |
https://hdl.handle.net/10356/144709 |
_version_ |
1688665625515261952 |