Vision-based robotic grasping: developing a grasp planning algorithm for object manipulation

• Main Author: Thio, Zheng Yang
• Other Authors: Chen I-Ming
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Vision
• Online Access: https://hdl.handle.net/10356/177510
• Institution: Nanyang Technological University

Grasp detection has become a pivotal aspect in robotic manipulation, allowing robots to identify specific points on an object for successful grasping. This report proposes a vision-based grasping algorithm capable of generating grasp poses on both single and multi-object scenarios. The proposed algorithm was first elaborated in detail, discussing its architecture and usage of the Generative Residual Convolutional neural network (CNN) as a backbone, its hyperparameters, loss function and evaluation metrics. Secondly, the steps required to collect the custom dataset were elaborated to show the complexity and quality of the dataset. Thirdly, the proposed algorithm was then trained against the Cornell grasping dataset and different variations of the custom dataset. The models trained on the different dataset were then compared based on the validation and evaluation metrics and the generated grasp pose of different objects, multi-objects and lastly novel objects. In general, the models produced satisfactory results but there were still limitations which was elaborated. This project was also integrated with another Final Year Project that utilised ROS2 to develop a motion planning control module for a UR10 robotic arm. The integration of these two projects were used to perform pick-and-place tasks on static objects and it results were discussed. The report then ends with a summary of the project, mentioning the overall progress of the project and the results of the proposed algorithm. Future developments were also elaborated with the aim of addressing the existing limitations