Hierarchical multi-agent reinforcement learning with options

• Main Author: Ang, Wan Qi
• Other Authors: Lana Obraztsova
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/148028
• Institution: Nanyang Technological University

In recent years, there are many state-of-the-art multi-agent reinforcement learning (MARL) algorithms that aim to get multiple agents to work together to achieve a common goal. COMA is one of these breakthroughs that proposes a counterfactual baseline to address the credit assignment problem in multi-agent scenarios. On the other hand, another research area known as hierarchical reinforcement learning (HRL) is gaining traction for its ability to model actions as a set of sub-actions to achieve temporal abstraction such as the options framework. This project aims to incorporate hierarchical reinforcement learning (HRL) frameworks into current state-of-the-art multi-agent reinforcement learning (MARL) algorithms through the use of the well-known options framework. As such, a novel MARL algorithm, COMA-OC, is introduced and evaluated on the SMAC benchmark in a variety of scenarios to analyse its performance and applicability. The results showed that COMA-OC is able to surpass the baseline's performance in most scenarios and has the potential to be developed further.