Enhancing DRL-based USV navigation with CVAE and gated graph convolutional networks

• Main Author: Deng, Haoyuan
• Other Authors: Jiang Xudong
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Conditional variational autoencoder; GatedGCN; Reinforcement learning; Maritime navigation; Unmanned surface vehicles
• Online Access: https://hdl.handle.net/10356/181856
• Institution: Nanyang Technological University

This thesis introduces a novel Conditional Variational Autoencoder (CVAE) integrated with a Gated Graph Convolutional Network (GatedGCN) for Reinforcement Learning (RL), specifically designed for the complex and dynamic environment of maritime navigation. The CVAE-GatedGCN-RL model is engineered to enhance the decision-making capabilities of Unmanned Surface Vehicles (USVs) by effectively learning and adapting navigational strategies to real-time environmental interactions and obstacles. By incorporating GatedGCN within the CVAE’s encoder networks, the model optimizes the processing of spatial and relational data, thereby achieving more effective state representation and decision-making, proving its superior performance over traditional RL methods. The study utilizes two vessel navigation datasets organized from real Automatic Identification System (AIS) data: ht1 and ht2, where the model undergoes strategy training and testing respectively. Comparative analysis with state-of-the-art RL techniques such as GCN-RL and MP-GatedGCN-RL demonstrates the advantages of integrating CVAE into the RL framework, particularly in terms of learning efficiency and operational success rates. The thesis concludes with potential future improvements, including the refinement of reward structures and policy networks, and enhancements to the CVAE, aimed at further advancing the model’s capabilities.