Programmatic policies for interpretable reinforcement learning using pre-trained models

Programmatic policies for interpretable reinforcement learning using pre-trained models

Decision Trees (DTs) are widely used in machine learning due to their critical interpretability. However, training DTs in a Reinforcement Learning (RL) setting is challenging. In the project, we present a framework to improve the interpretability of reinforcement learning (RL) by generating prog...

Full description

Saved in:
Bibliographic Details
Main Author: Tu, Xia Yang
Other Authors: Arvind Easwaran
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2024
Subjects:
Computer and Information Science
LLM
Reinforcement learning
Online Access:https://hdl.handle.net/10356/181169
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-181169
record_format dspace
spelling sg-ntu-dr.10356-1811692024-11-18T01:24:17Z Programmatic policies for interpretable reinforcement learning using pre-trained models Tu, Xia Yang Arvind Easwaran College of Computing and Data Science arvinde@ntu.edu.sg Computer and Information Science LLM Reinforcement learning Decision Trees (DTs) are widely used in machine learning due to their critical interpretability. However, training DTs in a Reinforcement Learning (RL) setting is challenging. In the project, we present a framework to improve the interpretability of reinforcement learning (RL) by generating programmatic policies through large language models (LLMs). Initially looking at decision trees within latent space representation, we proceed to develop and apply a “code reflection” framework in the Karel environment, which provides a controlled setting for evaluating task performance. The framework leverages prompt engineering and code reflection to optimize program synthesis, producing interpretable policies. The report discusses the findings and concludes with recommendations for future work at the end. Bachelor's degree 2024-11-18T01:24:16Z 2024-11-18T01:24:16Z 2024 Final Year Project (FYP) Tu, X. Y. (2024). Programmatic policies for interpretable reinforcement learning using pre-trained models. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/181169 https://hdl.handle.net/10356/181169 en SCSE23-0622 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Computer and Information Science
LLM
Reinforcement learning
spellingShingle Computer and Information Science
LLM
Reinforcement learning
Tu, Xia Yang
Programmatic policies for interpretable reinforcement learning using pre-trained models
description Decision Trees (DTs) are widely used in machine learning due to their critical interpretability. However, training DTs in a Reinforcement Learning (RL) setting is challenging. In the project, we present a framework to improve the interpretability of reinforcement learning (RL) by generating programmatic policies through large language models (LLMs). Initially looking at decision trees within latent space representation, we proceed to develop and apply a “code reflection” framework in the Karel environment, which provides a controlled setting for evaluating task performance. The framework leverages prompt engineering and code reflection to optimize program synthesis, producing interpretable policies. The report discusses the findings and concludes with recommendations for future work at the end.
author2 Arvind Easwaran
author_facet Arvind Easwaran
Tu, Xia Yang
format Final Year Project
author Tu, Xia Yang
author_sort Tu, Xia Yang
title Programmatic policies for interpretable reinforcement learning using pre-trained models
title_short Programmatic policies for interpretable reinforcement learning using pre-trained models
title_full Programmatic policies for interpretable reinforcement learning using pre-trained models
title_fullStr Programmatic policies for interpretable reinforcement learning using pre-trained models
title_full_unstemmed Programmatic policies for interpretable reinforcement learning using pre-trained models
title_sort programmatic policies for interpretable reinforcement learning using pre-trained models
publisher Nanyang Technological University
publishDate 2024
url https://hdl.handle.net/10356/181169
_version_ 1816859022167375872

Similar Items