Development of a robot vision system for object detection and localization

• Main Author: Ning, Shuqi
• Other Authors: CHEAH Chien Chern
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/141185
• Institution: Nanyang Technological University

With the increasing speed of computer operation, machine vision has a wide prospect in robot positioning and navigation, human-computer interaction, industrial measurement and other fields. This project aims to develop a robot vision system which involves object detections and object localization. The object detection using a fixed camera is often limited by the angle and filed in the environment and therefore it is difficult to detect a wide range of objects. The project used a Jidetech Outdoor 2MP Waterproof 20X Zoom POE PTZ camera, which can rotate 360 degrees to detect the whole environment. Due to its inability to detect distance, a vision algorithm combining depth camera and PTZ camera is proposed, and a robot vision system is developed to estimate the 3D depth information of the whole environment. The main work includes the following aspects. Firstly, the composition and principle of object detection algorithm under deep learning are studied, and YOLOv2 algorithm is selected as the object detection algorithm. Then, based on the darkflow framework, the collected data sets are trained, and the accuracy is ensured to meet the test requirements. Secondly, the working principle of D45i RGB-D camera is studied and it is used to detect and measure the object. Then, by controlling of the rotation angle of PTZ camera, the detection of object is realized under PTZ camera. According to the position of the object, two detection algorithms are developed to obtain the relative position form the camera with the object. Therefore, the RGB-D camera can be calibrated with respect to the PTZ camera to estimate the depth information so as to determine the location of the object. Finally, through the comparison of the experimental results, the error data is obtained and analyzed so as to improve the algorithms for localization of the object with better accuracy.