Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning

Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aeria...

Full description

Saved in:
Bibliographic Details
Main Authors: Zhang, Mingcheng, Yan, Chao, Dai, Wei, Xiang, Xiaojia, Low, Kin Huat
Other Authors: School of Mechanical and Aerospace Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/171060
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-171060
record_format dspace
spelling sg-ntu-dr.10356-1710602023-12-21T04:35:40Z Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning Zhang, Mingcheng Yan, Chao Dai, Wei Xiang, Xiaojia Low, Kin Huat School of Mechanical and Aerospace Engineering Air Traffic Management Research Institute Engineering::Aeronautical engineering::Aviation Unmanned Aircraft System Traffic Management Tactical Conflict Resolution Double Deep Q Network Attention Mechanism Secondary Conflict Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aerial systems (UASs) is an essential piece of the puzzle for the future UAS Traffic Management (UTM), especially in very low-level (VLL) urban airspace. Unlike conflict resolution in higher altitude airspace, the dense high-rise buildings are an essential source of potential conflict to be considered in VLL urban airspace. In this paper, we propose an attention-based deep reinforcement learning approach to solve the tactical conflict resolution problem. Specifically, we formulate this task as a sequential decision-making problem using Markov Decision Process (MDP). The double deep Q network (DDQN) framework is used as a learning framework for the host drone to learn to output conflict-free maneuvers at each time step. We use the attention mechanism to model the individual neighbor's effect on the host drone, endowing the learned conflict resolution policy to be adapted to an arbitrary number of neighboring drones. Lastly, we build a simulation environment with various scenarios covering different types of encounters to evaluate the proposed approach. The simulation results demonstrate that our proposed algorithm provides a reliable solution to minimize secondary conflict counts compared to learning and non-learning-based approaches under different traffic density scenarios. Civil Aviation Authority of Singapore (CAAS) Nanyang Technological University National Research Foundation (NRF) Published version This research is supported by the National Research Foundation (NRF), Singapore, and the Civil Aviation Authority of Singapore (CAAS), under the Aviation Transformation Programme (ATP). The Research Student Scholarship provided by Air Traffic Management Research Institute (ATMRI) to the first author is acknowledged. 2023-10-11T01:02:21Z 2023-10-11T01:02:21Z 2023 Journal Article Zhang, M., Yan, C., Dai, W., Xiang, X. & Low, K. H. (2023). Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning. Green Energy and Intelligent Transportation, 2(4), 100107-. https://dx.doi.org/10.1016/j.geits.2023.100107 2773-1537 https://hdl.handle.net/10356/171060 10.1016/j.geits.2023.100107 2-s2.0-85166231636 4 2 100107 en Green Energy and Intelligent Transportation © 2023 The Authors. Published by Elsevier Ltd on behalf of Beijing Institute of Technology Press Co., Ltd. This is an open access article under the CC BYNC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Aeronautical engineering::Aviation
Unmanned Aircraft System Traffic Management
Tactical Conflict Resolution
Double Deep Q Network
Attention Mechanism
Secondary Conflict
spellingShingle Engineering::Aeronautical engineering::Aviation
Unmanned Aircraft System Traffic Management
Tactical Conflict Resolution
Double Deep Q Network
Attention Mechanism
Secondary Conflict
Zhang, Mingcheng
Yan, Chao
Dai, Wei
Xiang, Xiaojia
Low, Kin Huat
Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
description Unmanned aerial vehicles (UAVs) have gained much attention from academic and industrial areas due to the significant number of potential applications in urban airspace. A traffic management system for these UAVs is needed to manage this future traffic. Tactical conflict resolution for unmanned aerial systems (UASs) is an essential piece of the puzzle for the future UAS Traffic Management (UTM), especially in very low-level (VLL) urban airspace. Unlike conflict resolution in higher altitude airspace, the dense high-rise buildings are an essential source of potential conflict to be considered in VLL urban airspace. In this paper, we propose an attention-based deep reinforcement learning approach to solve the tactical conflict resolution problem. Specifically, we formulate this task as a sequential decision-making problem using Markov Decision Process (MDP). The double deep Q network (DDQN) framework is used as a learning framework for the host drone to learn to output conflict-free maneuvers at each time step. We use the attention mechanism to model the individual neighbor's effect on the host drone, endowing the learned conflict resolution policy to be adapted to an arbitrary number of neighboring drones. Lastly, we build a simulation environment with various scenarios covering different types of encounters to evaluate the proposed approach. The simulation results demonstrate that our proposed algorithm provides a reliable solution to minimize secondary conflict counts compared to learning and non-learning-based approaches under different traffic density scenarios.
author2 School of Mechanical and Aerospace Engineering
author_facet School of Mechanical and Aerospace Engineering
Zhang, Mingcheng
Yan, Chao
Dai, Wei
Xiang, Xiaojia
Low, Kin Huat
format Article
author Zhang, Mingcheng
Yan, Chao
Dai, Wei
Xiang, Xiaojia
Low, Kin Huat
author_sort Zhang, Mingcheng
title Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
title_short Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
title_full Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
title_fullStr Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
title_full_unstemmed Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
title_sort tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learning
publishDate 2023
url https://hdl.handle.net/10356/171060
_version_ 1787136716349374464