Deep reinforcement learning for UAV routing in the presence of multiple charging stations

Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to reple...

Full description

Saved in:
Bibliographic Details
Main Authors: Fan, Mingfeng, Wu, Yaoxin, Liao, Tianjun, Cao, Zhiguang, Guo, Hongliang, Sartoretti, Guillaume, Wu, Guohua
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/170774
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-170774
record_format dspace
spelling sg-ntu-dr.10356-1707742023-10-02T07:32:33Z Deep reinforcement learning for UAV routing in the presence of multiple charging stations Fan, Mingfeng Wu, Yaoxin Liao, Tianjun Cao, Zhiguang Guo, Hongliang Sartoretti, Guillaume Wu, Guohua School of Computer Science and Engineering Engineering::Computer science and engineering Combinatorial Optimization Problems Deep Reinforcement Learning Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to replenish energy from charging stations.In this paper, we study a UAV routing problem in the presence of multiple charging stations (URPMCS) with the objective of minimizing the total distance traveled by the UAV during traffic monitoring. We present a deep reinforcement learning based method, where a multi-head heterogeneous attention mechanism is designed to facilitate learning a policy that automatically and sequentially constructs the route, while taking the energy consumption into account. In our method, two types of attentions are leveraged to learn the relations between monitoring targets and charging station nodes, adopting an encoder-decoder-like policy network. Moreover, we also employ a curriculum learning strategy to enhance generalization to different numbers of charging stations. Computational results show that our method outperforms conventional algorithms with higher solution quality (except for exact methods such as Gurobi) and shorter runtime in general, and also exhibits strong generalized performance on problem instances with different distributions and sizes. This work was supported in part by the National Natural Science Foundation of China under Grant 62073341 and in part by the Fundamental Research Funds for the Central Universities of Central South University under Grant 2022ZZTS0191. 2023-10-02T07:32:33Z 2023-10-02T07:32:33Z 2023 Journal Article Fan, M., Wu, Y., Liao, T., Cao, Z., Guo, H., Sartoretti, G. & Wu, G. (2023). Deep reinforcement learning for UAV routing in the presence of multiple charging stations. IEEE Transactions On Vehicular Technology, 72(5), 5732-5746. https://dx.doi.org/10.1109/TVT.2022.3232607 0018-9545 https://hdl.handle.net/10356/170774 10.1109/TVT.2022.3232607 2-s2.0-85146225594 5 72 5732 5746 en IEEE Transactions on Vehicular Technology © 2022 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Combinatorial Optimization Problems
Deep Reinforcement Learning
spellingShingle Engineering::Computer science and engineering
Combinatorial Optimization Problems
Deep Reinforcement Learning
Fan, Mingfeng
Wu, Yaoxin
Liao, Tianjun
Cao, Zhiguang
Guo, Hongliang
Sartoretti, Guillaume
Wu, Guohua
Deep reinforcement learning for UAV routing in the presence of multiple charging stations
description Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to replenish energy from charging stations.In this paper, we study a UAV routing problem in the presence of multiple charging stations (URPMCS) with the objective of minimizing the total distance traveled by the UAV during traffic monitoring. We present a deep reinforcement learning based method, where a multi-head heterogeneous attention mechanism is designed to facilitate learning a policy that automatically and sequentially constructs the route, while taking the energy consumption into account. In our method, two types of attentions are leveraged to learn the relations between monitoring targets and charging station nodes, adopting an encoder-decoder-like policy network. Moreover, we also employ a curriculum learning strategy to enhance generalization to different numbers of charging stations. Computational results show that our method outperforms conventional algorithms with higher solution quality (except for exact methods such as Gurobi) and shorter runtime in general, and also exhibits strong generalized performance on problem instances with different distributions and sizes.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Fan, Mingfeng
Wu, Yaoxin
Liao, Tianjun
Cao, Zhiguang
Guo, Hongliang
Sartoretti, Guillaume
Wu, Guohua
format Article
author Fan, Mingfeng
Wu, Yaoxin
Liao, Tianjun
Cao, Zhiguang
Guo, Hongliang
Sartoretti, Guillaume
Wu, Guohua
author_sort Fan, Mingfeng
title Deep reinforcement learning for UAV routing in the presence of multiple charging stations
title_short Deep reinforcement learning for UAV routing in the presence of multiple charging stations
title_full Deep reinforcement learning for UAV routing in the presence of multiple charging stations
title_fullStr Deep reinforcement learning for UAV routing in the presence of multiple charging stations
title_full_unstemmed Deep reinforcement learning for UAV routing in the presence of multiple charging stations
title_sort deep reinforcement learning for uav routing in the presence of multiple charging stations
publishDate 2023
url https://hdl.handle.net/10356/170774
_version_ 1779156472681201664