Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks
In this paper, we employ multiple UAVs coordinated by a base station (BS) to help the ground users (GUs) to offload their sensing data. Different UAVs can adapt their trajectories and network formation to expedite data transmissions via multi-hop relaying. The trajectory planning aims to collect all...
Saved in:
Main Authors: | , , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/170818 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-170818 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1708182023-10-03T05:52:49Z Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks Gong, Shimin Wang, Meng Gu, Bo Zhang, Wenjie Hoang, Dinh Thai Niyato, Dusit School of Computer Science and Engineering Engineering::Computer science and engineering Trajectory Planning Bayesian Optimization In this paper, we employ multiple UAVs coordinated by a base station (BS) to help the ground users (GUs) to offload their sensing data. Different UAVs can adapt their trajectories and network formation to expedite data transmissions via multi-hop relaying. The trajectory planning aims to collect all GUs' data, while the UAVs' network formation optimizes the multi-hop UAV network topology to minimize the energy consumption and transmission delay. The joint network formation and trajectory optimization is solved by a two-step iterative approach. Firstly, we devise the adaptive network formation scheme by using a heuristic algorithm to balance the UAVs' energy consumption and data queue size. Then, with the fixed network formation, the UAVs' trajectories are further optimized by using multi-agent deep reinforcement learning without knowing the GUs' traffic demands and spatial distribution. To improve the learning efficiency, we further employ Bayesian optimization to estimate the UAVs' flying decisions based on historical trajectory points. This helps avoid inefficient action explorations and improves the convergence rate in the model training. The simulation results reveal close spatial-temporal couplings between the UAVs' trajectory planning and network formation. Compared with several baselines, our solution can better exploit the UAVs' cooperation in data offloading, thus improving energy efficiency and delay performance. Info-communications Media Development Authority (IMDA) National Research Foundation (NRF) The work of Shimin Gong was supported in part by the National Natural Science Foundation of China under Grant 61972434 and in part by Shenzhen Fundamental Research Program under Grants JCYJ20220818103201004 and JCYJ20190807154009444. The work of Bo Gu was supported in part by the National Science Foundation of China under Grant U20A20175. The work of Dusit Niyato was supported in part by the National Research Foundation, Singapore, and in part by Infocomm Media Development Authority under the Future Communications Research Development Programme (FCP), and in part by Defence Science Organisation (DSO) National Laboratories under the AI Singapore Programme (AISG) under Grant AISG2-RP-2020-019, through Energy Research Test-Bed and Industry Partnership Funding Initiative, a part of the Energy Grid (EG) 2.0 Programme, and through DesCartes and the Campus for Research Excellence and Technological Enterprise (CREATE) Programme. 2023-10-03T05:52:48Z 2023-10-03T05:52:48Z 2023 Journal Article Gong, S., Wang, M., Gu, B., Zhang, W., Hoang, D. T. & Niyato, D. (2023). Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks. IEEE Transactions On Vehicular Technology, 72(8), 10933-10948. https://dx.doi.org/10.1109/TVT.2023.3262778 0018-9545 https://hdl.handle.net/10356/170818 10.1109/TVT.2023.3262778 2-s2.0-85151556086 8 72 10933 10948 en AISG2-RP-2020-019 IEEE Transactions on Vehicular Technology © 2023 IEEE. All rights reserved. |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering Trajectory Planning Bayesian Optimization |
spellingShingle |
Engineering::Computer science and engineering Trajectory Planning Bayesian Optimization Gong, Shimin Wang, Meng Gu, Bo Zhang, Wenjie Hoang, Dinh Thai Niyato, Dusit Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
description |
In this paper, we employ multiple UAVs coordinated by a base station (BS) to help the ground users (GUs) to offload their sensing data. Different UAVs can adapt their trajectories and network formation to expedite data transmissions via multi-hop relaying. The trajectory planning aims to collect all GUs' data, while the UAVs' network formation optimizes the multi-hop UAV network topology to minimize the energy consumption and transmission delay. The joint network formation and trajectory optimization is solved by a two-step iterative approach. Firstly, we devise the adaptive network formation scheme by using a heuristic algorithm to balance the UAVs' energy consumption and data queue size. Then, with the fixed network formation, the UAVs' trajectories are further optimized by using multi-agent deep reinforcement learning without knowing the GUs' traffic demands and spatial distribution. To improve the learning efficiency, we further employ Bayesian optimization to estimate the UAVs' flying decisions based on historical trajectory points. This helps avoid inefficient action explorations and improves the convergence rate in the model training. The simulation results reveal close spatial-temporal couplings between the UAVs' trajectory planning and network formation. Compared with several baselines, our solution can better exploit the UAVs' cooperation in data offloading, thus improving energy efficiency and delay performance. |
author2 |
School of Computer Science and Engineering |
author_facet |
School of Computer Science and Engineering Gong, Shimin Wang, Meng Gu, Bo Zhang, Wenjie Hoang, Dinh Thai Niyato, Dusit |
format |
Article |
author |
Gong, Shimin Wang, Meng Gu, Bo Zhang, Wenjie Hoang, Dinh Thai Niyato, Dusit |
author_sort |
Gong, Shimin |
title |
Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
title_short |
Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
title_full |
Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
title_fullStr |
Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
title_full_unstemmed |
Bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-UAV networks |
title_sort |
bayesian optimization enhanced deep reinforcement learning for trajectory planning and network formation in multi-uav networks |
publishDate |
2023 |
url |
https://hdl.handle.net/10356/170818 |
_version_ |
1779156790530801664 |