Deep reinforcement learning based navigation by goal-driven transformer

• Main Author: Wong, Yui
• Other Authors: Lyu Chen
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering
• Online Access: https://hdl.handle.net/10356/180704
• Institution: Nanyang Technological University

Notwithstanding the achievements in goal-driven navigation tasks, current approaches with the application of deep reinforcement learning are notably inefficient in data usage. A major cause of this inefficiency is the separation of goal information from the perceptual system, which is applied directly as a condition for decision-making. This separation allows non-goal-related features of the scene representation to negatively impact learning. To address this issue, a pioneering approach that integrates a goal-driven vision transformer with deep reinforcement learning was introduced. In this method, physical goal states are treated as inputs to a scene encoder to align the representation of scene with relevant goal information, thereby enabling an efficient performance in autonomous navigation. To be more specific, a modified version of vision transformer, named the goal-driven transformer, is developed as the core of the perception system, which uses pre-trained data generated from experts’ knowledge to enhance efficiency of data. By focusing on features that are directly relevant to the goal, the perception system is able to filter out irrelevant information and reduce data redundancy. This goaled approach not only improves the learning efficiency but accelerates the training process, enabling the system to adapt more rapidly to new scenarios. After that a deep reinforcement learning algorithm is applied to make decisions for action output. The perception system uses the representation of scene from the goal-driven transformer to produce decision commands. Consequently, the approach encourages the representation of real time scene to focus on features relevant to a given goal, significantly improving data efficiency and leading to superior navigation capability. Simulation tests have already been conducted to demonstrate the effectiveness of this integrated approach, which highlights the excellent performance in several critical areas. Moreover, the system exhibits notable improvements in data usage efficiency, robustness in handling unknown scenes and overall navigation performance. Such results underscore the potential of this approach to address existing limitations in autonomous navigation systems, and thus providing a more reliable and efficient solution with compared to current leading methods. By the method of focusing on goal-relevant features and leveraging expert knowledge, this innovative method marks a significant advancement in the field of goal-driven navigation.