Towards explaining sequences of actions in multi-agent deep reinforcement learning models

• Main Authors: KHAING, Phyo Wai, GENG, Minghong, SUBAGDJA, Budhitama, PATERIA, Shubham, TAN, Ah-hwee
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Multi Agent Deep Reinforcement Learning; Explainable Artificial Intelligence; Explainable Deep Reinforcement Learning; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8076; https://ink.library.smu.edu.sg/context/sis_research/article/9079/viewcontent/p2325.pdf
• Institution: Singapore Management University

Although Multi-agent Deep Reinforcement Learning (MADRL) has shown promising results in solving complex real-world problems, the applicability and reliability of MADRL models are often limited by a lack of understanding of their inner workings for explaining the decisions made. To address this issue, this paper proposes a novel method for explaining MADRL by generalizing the sequences of action events performed by agents into high-level abstract strategies using a spatio-temporal neural network model. Specifically, an interval-based memory retrieval procedure is developed to generalize the encoded sequences of action events over time into short sequential patterns. In addition, two abstraction algorithms are introduced, one for abstracting action events across multiple agents and the other for further abstracting the episodes over time into short sequential patterns, which can then be translated into symbolic form for interpretation. We evaluate the proposed method using the StarCraft Multi Agent Challenge (SMAC) benchmark task, which shows that the method is able to derive high-level explanations of MADRL models at various levels of granularity.