Development of an automatic fruit sorting robot based on object detection algorithm

• Main Author: Wang, Ke
• Other Authors: Cheah Chien Chern
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
• Online Access: https://hdl.handle.net/10356/157566
• Institution: Nanyang Technological University

In recent years, agricultural robots have becomes an increasingly popular filed, which bring huge development and innovation to the agriculture. Among these robot-related technologies, visual information is an important information for robots operating in unstructured environment. This project aims to design a banana sorting robot system base on an object detection neural network called YOLOv5(You Look Only Once) algorithm. Specifically, this project includes the preparation of the bananas dataset and use the dataset to train the YOLOv5 network. The image preprocessing is applied to acquire the binary image and find the picking point on the banana. Camera calibration and eye-hand calibration are also performed to obtain the coordinate of the picking point with respect to the robot base coordinate frame. A UR5e robot is used in this project and the RTDE(Real Time Data Exchange) package constructs the connection between the PC and the UR5e robot. Then a python code and its corresponding robot program are designed to operate the UR5e robot to complete the banana grabbing task. In the end, a practical experiment shows that the bananas can be successfully detected and picked up.