End to end autonomous driving simulation based on reinforcement learning

• Main Author: Wong, Kenzhi Iskandar
• Other Authors: Lyu Chen
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering
• Online Access: https://hdl.handle.net/10356/177154
• Institution: Nanyang Technological University

This paper presents a comprehensive study that explores the application of reinforcement learning (RL) algorithms, specifically Deep Q-Network (DQN) and Soft Actor Critic (SAC), in the context of end-to-end autonomous driving. The research project utilizes the SMARTS Simulator, an open-source software tailored for RL applications in autonomous driving scenarios. Employing an end-to-end approach, the research project utilizes RGB image inputs into an Artificial Neural Network, with image recognition facilitated by the Vision Transformer model. The study begins with a review of RL theory, covering key concepts such as the Markov Decision Process, exploration-exploitation strategies, policy, rewards, value functions, and the taxonomy of RL methods. Following this, the report introduces the DQN and SAC algorithms, providing insights into their applications in autonomous driving scenarios. Additionally, the study explores the Vision Transformer model for image recognition tasks within the autonomous driving domain. Subsequently, the paper outlines the experimental setup, detailing the environment, scenario, action space, reward structure, and termination conditions specific to the autonomous driving context. The design and implementation section clarifies the architectural framework and the specific implementation strategies within the SMARTS Simulator environment. Finally, the paper presents and discusses empirical results of applying both DQN and SAC algorithm from the experiments conducted.