Collaborative air-ground search with deep reinforcement learning

• Main Author: Lim, You Xuan
• Other Authors: Mir Feroskhan
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Engineering; Deep reinforcement learning; Intelligent agents; Collaborative search
• Online Access: https://hdl.handle.net/10356/177158
• Institution: Nanyang Technological University

Artificial intelligence (AI) has emerged as a leading area of research, particularly in the realm of training autonomous Unmanned Aerial Vehicles (UAVs). Target searching, a key focus within this domain, holds significant promise for applications such as runway approach, cargo pickup and delivery, and area surveillance. However, current target searching methods entail intensive path planning efforts to ensure precise drone actions. To address these complexities, reinforcement learning algorithms like Proximal Policy Optimization (PPO) have been employed to train autonomous behaviors. While PPO has shown promise, it lacks innate support for collaborative behaviors crucial to the success of drone swarms. In response to these challenges, this study extends beyond UAV training to incorporate Unmanned Ground Vehicles (UGVs), recognizing the limitations faced by each platform individually. Navigating environments solely with UGVs is hindered by limited observational capabilities, while relying solely on UAVs presents challenges in carrying heavy loads during search and rescue missions. To address these limitations, a dual-agent approach is adopted, training UAVs to locate targets and plan paths while leading UGVs. This integrated strategy effectively addresses navigational and load-bearing challenges, optimizing performance in real-world scenarios. In this study, we propose a Cooperative Multi-goal Multi-stage Multi-agent (CM3) Deep Reinforcement Learning approach for target search missions, aiming to address the inherent complexities and challenges in coordinating UAVs and UGVs effectively. By incorporating the CM3 framework, our approach facilitates seamless collaboration between multiple agents across different stages of the mission, thereby enhancing overall performance and efficiency. Training strategies using PPO and the Multi-Agent Posthumous Credit Assignment (MA-POCA) algorithm are also investigated to foster collaborative behaviors within the drone swarm. Notably, the study reveals insights into the impact of camera resolution on target search effectiveness and the importance of careful consideration when implementing group reward mechanisms. Additionally, the study highlights the necessity of aligning group reward assignments with agents' training stages in multi-stage reinforcement learning scenarios. In summary, this research underscores the significance of AI in enhancing UAV and UGV capabilities for target searching missions. By leveraging reinforcement learning algorithms and adopting a collaborative approach, the study offers valuable insights into optimizing performance and addressing challenges in real-world applications.