Intelligent robot manipulation with deep learning

The application of reinforcement learning (RL) in robotics has seen significant advancements across various sectors, yet a critical challenge persists: the simulation-to-reality (sim-to-real) gap. In real-world scenarios, robots frequently underperform due to their inability to access or accurate...

Full description

Saved in:
Bibliographic Details
Main Author: Tan, Jun Aun
Other Authors: Lin Zhiping
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2024
Subjects:
Online Access:https://hdl.handle.net/10356/176388
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-176388
record_format dspace
spelling sg-ntu-dr.10356-1763882024-05-17T15:43:49Z Intelligent robot manipulation with deep learning Tan, Jun Aun Lin Zhiping School of Electrical and Electronic Engineering A*STAR SIMTech EZPLin@ntu.edu.sg Engineering Reinforcement learning Imitation learning Robotics The application of reinforcement learning (RL) in robotics has seen significant advancements across various sectors, yet a critical challenge persists: the simulation-to-reality (sim-to-real) gap. In real-world scenarios, robots frequently underperform due to their inability to access or accurately interpret all observable states, such as the precise state of objects in their environment. This gap often results in discrepancies between expected and actual robot behavior, hindering the effective translation of learned skills from simulated environments to practical applications. This paper presents an innovative approach to bridge this gap through a combination of reinforcement learning (RL) and imitation learning (IL). We introduce a novel teacher-student framework designed to enhance the performance of robotic systems. In this framework, the teacher possesses complete access to all states, including both the environment and the robot, whereas the student is limited to observing only the robot's state and visual input. By implementing an imitation loss in conjunction with the Proximal Policy Optimization (PPO) policy loss during the training of the student's policy, and leveraging expert knowledge transferred from the teacher, we demonstrate a significant improvement in the student's learning efficiency and performance. Our results reveal that the student model, when trained under this hybrid learning paradigm, converges more swiftly, and outperforms models trained solely on RL. The experiments were conducted across multiple environments, each selected for their unique challenges and the ability to test the algorithm's adaptability and performance in varied conditions. This study not only showcases the potential of combining RL with imitation learning to mitigate the knowledge gap but also establishes a foundational framework for future research in enhancing the adaptability and efficiency of robotic systems in real-world applications. Bachelor's degree 2024-05-16T05:24:25Z 2024-05-16T05:24:25Z 2024 Final Year Project (FYP) Tan, J. A. (2024). Intelligent robot manipulation with deep learning. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/176388 https://hdl.handle.net/10356/176388 en B3112-231 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering
Reinforcement learning
Imitation learning
Robotics
spellingShingle Engineering
Reinforcement learning
Imitation learning
Robotics
Tan, Jun Aun
Intelligent robot manipulation with deep learning
description The application of reinforcement learning (RL) in robotics has seen significant advancements across various sectors, yet a critical challenge persists: the simulation-to-reality (sim-to-real) gap. In real-world scenarios, robots frequently underperform due to their inability to access or accurately interpret all observable states, such as the precise state of objects in their environment. This gap often results in discrepancies between expected and actual robot behavior, hindering the effective translation of learned skills from simulated environments to practical applications. This paper presents an innovative approach to bridge this gap through a combination of reinforcement learning (RL) and imitation learning (IL). We introduce a novel teacher-student framework designed to enhance the performance of robotic systems. In this framework, the teacher possesses complete access to all states, including both the environment and the robot, whereas the student is limited to observing only the robot's state and visual input. By implementing an imitation loss in conjunction with the Proximal Policy Optimization (PPO) policy loss during the training of the student's policy, and leveraging expert knowledge transferred from the teacher, we demonstrate a significant improvement in the student's learning efficiency and performance. Our results reveal that the student model, when trained under this hybrid learning paradigm, converges more swiftly, and outperforms models trained solely on RL. The experiments were conducted across multiple environments, each selected for their unique challenges and the ability to test the algorithm's adaptability and performance in varied conditions. This study not only showcases the potential of combining RL with imitation learning to mitigate the knowledge gap but also establishes a foundational framework for future research in enhancing the adaptability and efficiency of robotic systems in real-world applications.
author2 Lin Zhiping
author_facet Lin Zhiping
Tan, Jun Aun
format Final Year Project
author Tan, Jun Aun
author_sort Tan, Jun Aun
title Intelligent robot manipulation with deep learning
title_short Intelligent robot manipulation with deep learning
title_full Intelligent robot manipulation with deep learning
title_fullStr Intelligent robot manipulation with deep learning
title_full_unstemmed Intelligent robot manipulation with deep learning
title_sort intelligent robot manipulation with deep learning
publisher Nanyang Technological University
publishDate 2024
url https://hdl.handle.net/10356/176388
_version_ 1800916141911048192