Hierarchical multi-agent reinforcement learning with options - Ang, Wan Qi (Nanyang Technological University, 2021)
In recent years, there are many state-of-the-art multi-agent reinforcement learning (MARL) algorithms that aim to get multiple agents to work together to achieve a common goal. COMA is one of these breakthroughs that proposes a counterfactual baseline to address the credit assignment problem in multi-agent scenarios. On the other hand, another research area known as hierarchical reinforcement learning (HRL) is gaining traction for its ability to model actions as a set of sub-actions to achieve temporal abstraction such as the options framework. This project aims to incorporate hierarchical reinforcement learning (HRL) frameworks into current state-of-the-art multi-agent reinforcement learning (MARL) algorithms through the use of the well-known options framework. As such, a novel MARL algorithm, COMA-OC, is introduced and evaluated on the SMAC benchmark in a variety of scenarios to analyse its performance and applicability. The results showed that COMA-OC is able to surpass the baseline's performance in most scenarios and has the potential to be developed further.