Learning transferable skills in complex 3D scenarios via deep reinforcement learning

Deep Reinforcement Learning combines reinforcement learning, the framework that assists an intelligent agent towards its goal, with a deep neural network. The deep neural network follows a black-box model, performing complex functional approximation calculations to achieve the best results by minimi...

Full description

Saved in:
Bibliographic Details
Main Author: Lim, You Rong
Other Authors: Bo An
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2022
Subjects:
Online Access:https://hdl.handle.net/10356/156376
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-156376
record_format dspace
spelling sg-ntu-dr.10356-1563762022-04-15T08:49:33Z Learning transferable skills in complex 3D scenarios via deep reinforcement learning Lim, You Rong Bo An School of Computer Science and Engineering boan@ntu.edu.sg Engineering::Computer science and engineering Deep Reinforcement Learning combines reinforcement learning, the framework that assists an intelligent agent towards its goal, with a deep neural network. The deep neural network follows a black-box model, performing complex functional approximation calculations to achieve the best results by minimizing the output error through back-propagation. This process is both time and computationally expensive as it could take millions of iterations for the agent to master complex tasks. Recent success in Transfer Learning with Deep Reinforcement Learning has demonstrated the capability of jump-starting the learning process, resulting in better overall performance. Additionally, using previously attained knowledge can allow the agent to achieve minimum threshold performance in fewer training steps. These successes reduce the training steps required to master a complex task, saving computation resources and time. Therefore, my contributions would be investigating Deep Reinforcement Learning and its ability to learn and apply transferable skills within a complex environment involving sparse rewards and domain randomization through Transfer Learning. The study includes attaining transferable skills with Curriculum Learning and Reward Shaping to tackle the sparse rewards problem. Popular reinforcement learning algorithms Proximal Policy Optimisation (PPO) and Soft Actor-Critic(SAC) enabled the agent to learn the policy required to pass the minimum threshold. Following that, Transfer Learning was performed on the agent and trained in new scenarios. These experiments evaluate the capability of the policy to generalize a problem and encourage the agent to alter its existing policy under the new settings. The new settings involved inclined surfaces and changing the agent shape from ovoid to cubic. The results demonstrate that agent with transfer learning outperforms the untrained model under various metrics where the agent successfully adapted to changes by grasping observations without external interference. Bachelor of Engineering (Computer Science) 2022-04-15T08:49:33Z 2022-04-15T08:49:33Z 2022 Final Year Project (FYP) Lim, Y. R. (2022). Learning transferable skills in complex 3D scenarios via deep reinforcement learning. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/156376 https://hdl.handle.net/10356/156376 en SCSE21-0094 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
spellingShingle Engineering::Computer science and engineering
Lim, You Rong
Learning transferable skills in complex 3D scenarios via deep reinforcement learning
description Deep Reinforcement Learning combines reinforcement learning, the framework that assists an intelligent agent towards its goal, with a deep neural network. The deep neural network follows a black-box model, performing complex functional approximation calculations to achieve the best results by minimizing the output error through back-propagation. This process is both time and computationally expensive as it could take millions of iterations for the agent to master complex tasks. Recent success in Transfer Learning with Deep Reinforcement Learning has demonstrated the capability of jump-starting the learning process, resulting in better overall performance. Additionally, using previously attained knowledge can allow the agent to achieve minimum threshold performance in fewer training steps. These successes reduce the training steps required to master a complex task, saving computation resources and time. Therefore, my contributions would be investigating Deep Reinforcement Learning and its ability to learn and apply transferable skills within a complex environment involving sparse rewards and domain randomization through Transfer Learning. The study includes attaining transferable skills with Curriculum Learning and Reward Shaping to tackle the sparse rewards problem. Popular reinforcement learning algorithms Proximal Policy Optimisation (PPO) and Soft Actor-Critic(SAC) enabled the agent to learn the policy required to pass the minimum threshold. Following that, Transfer Learning was performed on the agent and trained in new scenarios. These experiments evaluate the capability of the policy to generalize a problem and encourage the agent to alter its existing policy under the new settings. The new settings involved inclined surfaces and changing the agent shape from ovoid to cubic. The results demonstrate that agent with transfer learning outperforms the untrained model under various metrics where the agent successfully adapted to changes by grasping observations without external interference.
author2 Bo An
author_facet Bo An
Lim, You Rong
format Final Year Project
author Lim, You Rong
author_sort Lim, You Rong
title Learning transferable skills in complex 3D scenarios via deep reinforcement learning
title_short Learning transferable skills in complex 3D scenarios via deep reinforcement learning
title_full Learning transferable skills in complex 3D scenarios via deep reinforcement learning
title_fullStr Learning transferable skills in complex 3D scenarios via deep reinforcement learning
title_full_unstemmed Learning transferable skills in complex 3D scenarios via deep reinforcement learning
title_sort learning transferable skills in complex 3d scenarios via deep reinforcement learning
publisher Nanyang Technological University
publishDate 2022
url https://hdl.handle.net/10356/156376
_version_ 1731235705894469632