Reinforcement learning for robot assembly
Robotic systems are traditionally employed in manufacturing to automate repetitive tasks such as welding, painting, and pick-and-place. Despite tremendous progress in robotics research, the classical assembly skill remains a challenge. In most cases, the difficult assembly skills still rely heavily...
Saved in:
Main Author: | |
---|---|
Other Authors: | |
Format: | Thesis-Doctor of Philosophy |
Language: | English |
Published: |
Nanyang Technological University
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/174724 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-174724 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1747242024-05-03T02:58:53Z Reinforcement learning for robot assembly Vuong Quoc Nghia Pham Quang Cuong School of Mechanical and Aerospace Engineering Singapore Centre for 3D Printing cuong@ntu.edu.sg Engineering Robotics Reinforcement learning Robotic systems are traditionally employed in manufacturing to automate repetitive tasks such as welding, painting, and pick-and-place. Despite tremendous progress in robotics research, the classical assembly skill remains a challenge. In most cases, the difficult assembly skills still rely heavily on the engineer’s expertise [1]. In addition, the skills are prone to failure in the face of new tasks or variations, such as the shape or size of objects. This is particularly important as customer demand for greater product variety has recently increased. Learning approaches will become prominent in this context since learning shifts the burden from humans to the robot. Instead of attempting to obtain an accurate model of the surrounding environments or to program the controller, the robot can acquire a dynamics model or directly learn optimal control policies from experience. Reinforcement Learning endows a robot with the ability to find optimal behavior autonomously by interacting with its surrounding environment. The integration of deep learning models into RL, known as deep reinforcement learning, has gained significant traction and demonstrated remarkable achievements across various domains. However, contemporary deep reinforcement learning algorithms still encounter numerous challenges when applied in real-world robot manipulation. First, samples on a robotics system are expensive and tedious to obtain. Adding to this problem, model-free deep reinforcement learning algorithms are known to be sample inefficient, i.e., they require a large number of samples. Second, real-world training raises safety concerns. The environment or the engineer might impose several constraints that the robot must satisfy at all times to ensure safety. These constraints are difficult to maintain during the exploration phase, which often involves random action sampling. The two mentioned challenges are among the fundamental issues that prevent integrating deep reinforcement learning into robotics control systems. This thesis demonstrates how we can possibly improve sample efficiency and enable safe learning, making RL more practical for realistic robot tasks. Firstly, it demonstrates substantial improvement in sample efficiency by using manipulation primitives as actions. Manipulation primitives are simple yet generic enough to generalize across various tasks. Secondly, incorporating low-level feedback controllers into RL provides prior knowledge, which can increase learning speed and improve policy performance. A key message in this work is that a robust and high-performance low-level controller can further improve the robustness and performance of policies. Finally, this thesis examines methods to narrow the reality gap - the fundamental problem in sim-to-real reinforcement learning. This work proposes a novel contact reduction method to improve simulation accuracy, facilitating sim-to-real transfer for complex assembly tasks. Doctor of Philosophy 2024-04-08T06:42:07Z 2024-04-08T06:42:07Z 2024 Thesis-Doctor of Philosophy Vuong Quoc Nghia (2024). Reinforcement learning for robot assembly. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/174724 https://hdl.handle.net/10356/174724 10.32657/10356/174724 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering Robotics Reinforcement learning |
spellingShingle |
Engineering Robotics Reinforcement learning Vuong Quoc Nghia Reinforcement learning for robot assembly |
description |
Robotic systems are traditionally employed in manufacturing to automate repetitive tasks such as welding, painting, and pick-and-place. Despite tremendous progress in robotics research, the classical assembly skill remains a challenge. In most cases, the difficult assembly skills still rely heavily on the engineer’s expertise [1]. In addition, the skills are prone to failure in the face of new tasks or variations, such as the shape or size of objects. This is particularly important as customer demand for greater product variety has recently increased. Learning approaches will become prominent in this context since learning shifts the burden from humans to the robot. Instead of attempting to obtain an accurate model of the surrounding environments or to program the controller, the robot can acquire a dynamics model or directly learn optimal control policies from experience. Reinforcement Learning endows a robot with the ability to find optimal behavior autonomously by interacting with its surrounding environment. The integration of deep learning models into RL, known as deep reinforcement learning, has gained significant traction and demonstrated remarkable achievements across various domains. However, contemporary deep reinforcement learning algorithms still encounter numerous challenges when applied in real-world robot manipulation. First, samples on a robotics system are expensive and tedious to obtain. Adding to this problem, model-free deep reinforcement learning algorithms are known to be sample inefficient, i.e., they require a large number of samples. Second, real-world training raises safety concerns. The environment or the engineer might impose several constraints that the robot must satisfy at all times to ensure safety. These constraints are difficult to maintain during the exploration phase, which often involves random action sampling. The two mentioned challenges are among the fundamental issues that prevent integrating deep reinforcement learning into robotics control systems. This thesis demonstrates how we can possibly improve sample efficiency and enable safe learning, making RL more practical for realistic robot tasks. Firstly, it demonstrates substantial improvement in sample efficiency by using manipulation primitives as actions. Manipulation primitives are simple yet generic enough to generalize across various tasks. Secondly, incorporating low-level feedback controllers into RL provides prior knowledge, which can increase learning speed and improve policy performance. A key message in this work is that a robust and high-performance low-level controller can further improve the robustness and performance of policies. Finally, this thesis examines methods to narrow the reality gap - the fundamental problem in sim-to-real reinforcement learning. This work proposes a novel contact reduction method to improve simulation accuracy, facilitating sim-to-real transfer for complex assembly tasks. |
author2 |
Pham Quang Cuong |
author_facet |
Pham Quang Cuong Vuong Quoc Nghia |
format |
Thesis-Doctor of Philosophy |
author |
Vuong Quoc Nghia |
author_sort |
Vuong Quoc Nghia |
title |
Reinforcement learning for robot assembly |
title_short |
Reinforcement learning for robot assembly |
title_full |
Reinforcement learning for robot assembly |
title_fullStr |
Reinforcement learning for robot assembly |
title_full_unstemmed |
Reinforcement learning for robot assembly |
title_sort |
reinforcement learning for robot assembly |
publisher |
Nanyang Technological University |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/174724 |
_version_ |
1814047331886039040 |