Hierarchical control of multi-agent reinforcement learning team in real-time strategy (RTS) games

• Main Authors: ZHOU, Weigui Jair, SUBAGDJA, Budhitama, TAN, Ah-hwee, ONG, Darren Wee Sze
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Hierarchical control; Self-organizing neural networks; Reinforcement learning; Real-time strategy games; Artificial Intelligence and Robotics; Numerical Analysis and Scientific Computing
• Online Access: https://ink.library.smu.edu.sg/sis_research/6242; https://ink.library.smu.edu.sg/context/sis_research/article/7245/viewcontent/manuscript_ESWA_S_21_0154_R1_27062021.pdf
• Institution: Singapore Management University

Coordinated control of multi-agent teams is an important task in many real-time strategy (RTS) games. In most prior work, micromanagement is the commonly used strategy whereby individual agents operate independently and make their own combat decisions. On the other extreme, some employ a macromanagement strategy whereby all agents are controlled by a single decision model. In this paper, we propose a hierarchical command and control architecture, consisting of a single high-level and multiple low-level reinforcement learning agents operating in a dynamic environment. This hierarchical model enables the low-level unit agents to make individual decisions while taking commands from the high-level commander agent. Compared with prior approaches, the proposed model provides the benefits of both flexibility and coordinated control. The performance of such hierarchical control model is demonstrated through empirical experiments in a real-time strategy game known as StarCraft: Brood War (SCBW).