Learning-enabled decision-making for autonomous driving: framework and methodology

• Main Author: Huang, Zhiyu
• Other Authors: Lyu Chen
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Mechanical engineering
• Online Access: https://hdl.handle.net/10356/172842
• Institution: Nanyang Technological University

The growing adoption of autonomous vehicles (AVs) holds the promise of transforming transportation systems, enhancing traffic safety, and supporting environmental sustainability. Despite significant advancements in advanced perception systems for AVs, the effectiveness of these vehicles in real-world scenarios depends critically on advanced decision-making systems. Therefore, a critical challenge lies in developing safe, robust, and human-like decision-making systems for achieving higher levels of vehicle autonomy. Harnessing the power of artificial intelligence (AI) and machine learning (ML) is pivotal in developing such advanced decision-making systems. These techniques offer promising solutions for tackling key challenges in autonomous driving systems, including scalability, adaptability, and alignment with human driver behaviors. This thesis presents a comprehensive framework and a series of learning-based methodologies for decision-making in AVs, with the objective of improving the scalability, adaptability, and alignment of their decision-making systems. The learning-enabled decision-making framework consists of three essential modules: world model, actor, and cost. These modules work collaboratively to provide a unified solution for autonomous machine intelligence and can be learned individually or jointly from human driving data. The methodologies proposed in this thesis focus on three key areas: (1) creating ML-driven actors and cost functions that align with human values, (2) developing ML-based world models capable of accurately forecasting complex human interactions, and (3) designing ML-based decision-making architectures that seamlessly integrate these components. Initially, we explore each module individually. For the actor module, we propose an approach based on reinforcement learning (RL) that incorporates human prior knowledge into the training process. By combining imitation learning (IL) from human expert demonstrations with RL, a novel learning framework is developed to align agent behavior with human expectations. The results show improved sample efficiency, enabling the agent to navigate complex urban environments safely, while also generating distinct driving styles that closely resemble human driving behaviors. In the cost module, we focus on learning personalized cost functions underlying human driving behaviors using inverse reinforcement learning (IRL). We propose a maximum entropy IRL algorithm that infers the reward function based on a structural assumption about discrete latent intentions guiding continuous low-level control actions. The personalized cost learning method outperforms general cost modeling methods, leading to a more human-like driving experience. Regarding the world model module, the goal is to accurately represent the complexity and dynamics of human behaviors in real-world driving environments. We introduce a joint multi-agent prediction model based on Transformers, leveraging hierarchical game theory to model interactions among agents in driving scenarios. The model employs a novel Transformer decoder structure and a learning process that regulates agent behavior to respond to other agents' behaviors. Extensive experiments validate the model's state-of-the-art prediction accuracy and its capability to jointly reason about ego agent motion plans and other agents' behaviors. Following individual assessments, unified predictive decision-making architectures are presented, harmonizing the three modules. A comprehensive behavior planning framework that integrates all three core modules is proposed. It generates diverse trajectory proposals, forecasts multi-modal futures for other agents, and evaluates candidate plans using a learned cost function. Importantly, this framework includes a conditional motion predictor (CMP) that forecasts other agents' responses to the potential actions of the AV. The CMP improves prediction accuracy and facilitates the selection of human-like behaviors. Moreover, we propose a differentiable and integrated prediction-planning framework that enables end-to-end training of the entire decision-making system. The framework incorporates a differentiable nonlinear optimizer as the motion planner, allowing the gradients to flow between the three core modules and directly optimizing the planning performance. The experimental results demonstrate superior performance in both open-loop and closed-loop testing, effectively handling complex urban driving scenarios. This thesis not only makes significant contributions to the advancement of autonomous driving systems but also lays the foundation for future research in learning-based decision-making for a wide range of intelligent systems. It paves the way for safe and human-centered autonomy, while driving the evolution of intelligent transportation systems.