Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aeria...
Saved in:
Main Authors: | , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/171060 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-171060 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1710602023-12-21T04:35:40Z Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning Zhang, Mingcheng Yan, Chao Dai, Wei Xiang, Xiaojia Low, Kin Huat School of Mechanical and Aerospace Engineering Air Traffic Management Research Institute Engineering::Aeronautical engineering::Aviation Unmanned Aircraft System Traffic Management Tactical Conflict Resolution Double Deep Q Network Attention Mechanism Secondary Conflict Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aerial systems (UASs) is an essential piece of the puzzle for the future UAS Traffic Management (UTM), especially in very low-level (VLL) urban airspace. Unlike conflict resolution in higher altitude airspace, the dense high-rise buildings are an essential source of potential conflict to be considered in VLL urban airspace. In this paper, we propose an attention-based deep reinforcement learning approach to solve the tactical conflict resolution problem. Specifically, we formulate this task as a sequential decision-making problem using Markov Decision Process (MDP). The double deep Q network (DDQN) framework is used as a learning framework for the host drone to learn to output conflict-free maneuvers at each time step. We use the attention mechanism to model the individual neighbor's effect on the host drone, endowing the learned conflict resolution policy to be adapted to an arbitrary number of neighboring drones. Lastly, we build a simulation environment with various scenarios covering different types of encounters to evaluate the proposed approach. The simulation results demonstrate that our proposed algorithm provides a reliable solution to minimize secondary conflict counts compared to learning and non-learning-based approaches under different traffic density scenarios. Civil Aviation Authority of Singapore (CAAS) Nanyang Technological University National Research Foundation (NRF) Published version This research is supported by the National Research Foundation (NRF), Singapore, and the Civil Aviation Authority of Singapore (CAAS), under the Aviation Transformation Programme (ATP). The Research Student Scholarship provided by Air Traffic Management Research Institute (ATMRI) to the first author is acknowledged. 2023-10-11T01:02:21Z 2023-10-11T01:02:21Z 2023 Journal Article Zhang, M., Yan, C., Dai, W., Xiang, X. & Low, K. H. (2023). Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning. Green Energy and Intelligent Transportation, 2(4), 100107-. https://dx.doi.org/10.1016/j.geits.2023.100107 2773-1537 https://hdl.handle.net/10356/171060 10.1016/j.geits.2023.100107 2-s2.0-85166231636 4 2 100107 en Green Energy and Intelligent Transportation © 2023 The Authors. Published by Elsevier Ltd on behalf of Beijing Institute of Technology Press Co., Ltd. This is an open access article under the CC BYNC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Aeronautical engineering::Aviation Unmanned Aircraft System Traffic Management Tactical Conflict Resolution Double Deep Q Network Attention Mechanism Secondary Conflict |
spellingShingle |
Engineering::Aeronautical engineering::Aviation Unmanned Aircraft System Traffic Management Tactical Conflict Resolution Double Deep Q Network Attention Mechanism Secondary Conflict Zhang, Mingcheng Yan, Chao Dai, Wei Xiang, Xiaojia Low, Kin Huat Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
description |
Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aerial systems (UASs) is an essential piece of the puzzle for the future UAS Traffic Management (UTM), especially in very low-level (VLL) urban airspace. Unlike conflict resolution in higher altitude airspace, the dense high-rise buildings are an essential source of potential conflict to be considered in VLL urban airspace. In this paper, we propose an attention-based deep reinforcement learning approach to solve the tactical conflict resolution problem. Specifically, we formulate this task as a sequential decision-making problem using Markov Decision Process (MDP). The double deep Q network (DDQN) framework is used as a learning framework for the host drone to learn to output conflict-free maneuvers at each time step. We use the attention mechanism to model the individual neighbor's effect on the host drone, endowing the learned conflict resolution policy to be adapted to an arbitrary number of neighboring drones. Lastly, we build a simulation environment with various scenarios covering different types of encounters to evaluate the proposed approach. The simulation results demonstrate that our proposed algorithm provides a reliable solution to minimize secondary conflict counts compared to learning and non-learning-based approaches under different traffic density scenarios. |
author2 |
School of Mechanical and Aerospace Engineering |
author_facet |
School of Mechanical and Aerospace Engineering Zhang, Mingcheng Yan, Chao Dai, Wei Xiang, Xiaojia Low, Kin Huat |
format |
Article |
author |
Zhang, Mingcheng Yan, Chao Dai, Wei Xiang, Xiaojia Low, Kin Huat |
author_sort |
Zhang, Mingcheng |
title |
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
title_short |
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
title_full |
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
title_fullStr |
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
title_full_unstemmed |
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
title_sort |
tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning |
publishDate |
2023 |
url |
https://hdl.handle.net/10356/171060 |
_version_ |
1787136716349374464 |