Guaranteed hierarchical reinforcement learning

• Main Author: Ang, Riley Xile
• Other Authors: Arvind Easwaran
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Reinforcement learning; PAC; Hierarchical reinforcement learning; Q-learning
• Online Access: https://hdl.handle.net/10356/175473
• Institution: Nanyang Technological University

Reinforcement learning (RL) is a sub-field of machine learning that aims to train an agent in an interactive environment to sequentially make choices via a process of trial-and-error, to maximize a total reward over time. RL has been studied for decades and has a strong and established theoretical foundation. Practically, it has gained prominence owing to projects in a wide range of fields including gaming, robotics, automation, etc. Despite its contributions and rise to popularity, RL is often resource-intensive in both its training time and memory requirements. Successfully training an agent with low margin of errors and high confidence bounds continues to remain an open research problem. Consequently, the focus of this project will be to use existing RL algorithms, particularly Speedy Q-Learning (SQL), a variant of tabular model-free Q-Learning, to design a Hierarchical Reinforcement Learning (HRL) agent in a continuous state space setting. Additionally, this project aims to evaluate the overall performance of the agent against proven theoretical bounds with the Probably Approximately Correct (PAC) framework.