Hierarchical reinforcement learning: A comprehensive survey

• Main Authors: PATERIA, Shubham, SUBAGDJA, Budhitama, TAN, Ah-hwee, QUEK, Chai
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Databases and Information Systems; Data Storage Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/6047; https://ink.library.smu.edu.sg/context/sis_research/article/7054/viewcontent/a109_pateria_suppl.pdf
• Institution: Singapore Management University

Hierarchical Reinforcement Learning (HRL) enables autonomous decomposition of challenging long-horizon decision-making tasks into simpler subtasks. During the past years, the landscape of HRL research has grown profoundly, resulting in copious approaches. A comprehensive overview of this vast landscape is necessary to study HRL in an organized manner. We provide a survey of the diverse HRL approaches concerning the challenges of learning hierarchical policies, subtask discovery, transfer learning, and multi-agent learning using HRL. The survey is presented according to a novel taxonomy of the approaches. Based on the survey, a set of important open problems is proposed to motivate the future research in HRL. Furthermore, we outline a few suitable task domains for evaluating the HRL approaches and a few interesting examples of the practical applications of HRL in the Supplementary Material.