Trust-region inverse reinforcement learning

• المؤلفون الرئيسيون: Cao, Kun, Xie, Lihua
• مؤلفون آخرون: School of Electrical and Electronic Engineering
• التنسيق: مقال
• اللغة: English
• منشور في: 2023
• الموضوعات: Engineering::Electrical and electronic engineering; Trust Region Methods; PMP
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/170705
• المؤسسة: Nanyang Technological University

This paper proposes a new unified inverse reinforcement learning (IRL) framework based on trust-region methods and a recently proposed Pontryagin differential programming (PDP) method in Jin et al. (2020), which aims to learn the parameters in both the system model and the cost function for three types of problems, namely, N-player nonzero-sum multistage games, 2-player zero-sum multistage games and 1-player optimal control, from demonstrated trajectories. Different from the existing frameworks using gradient to update learning parameters, our framework updates them with the candidate solution of trust-region subproblem (TRS), where its required gradient and Hessian are obtained by differentiating Pontryagin's Maximum Principle (PMP) equations once and twice, respectively. The differentiated equations are shown to be equivalent to the PMP equations for affine-quadratic games / optimal control problems and can be solved by some explicit recursions. Extensive simulation examples and comparisons are presented to demonstrate the effectiveness of our proposed algorithm.