Pallet detection and localization in warehouse environment

• Main Author: Zia, Saeedeh
• Other Authors: L. G. Pee
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Engineering::Computer science and engineering::Information systems
• Online Access: https://hdl.handle.net/10356/144538
• Institution: Nanyang Technological University

Autonomous warehousing operations are important target elements of Industry 4.0. One major component of autonomous warehousing is the automated cargo handling system that significantly contributes to high-velocity distribution of goods, which directly and substantially contributes to cost reduction in warehousing procedure. Robotic cargo handling is a highly flexible solution to realize the automated cargo handling system. One of the major bottlenecks of robotic cargo handling is the detection of elements in warehouse environment using visual perception sensors. In this thesis, the research on detection of pallet and cargo in the warehouse environment using deep learning techniques is reported, and the entire process from data generation to network training is explained in detail. This work is a partial of an on-going research on robotic warehousing, which was started in our research group about three years ago and it is expected to continue for several years after the submission of this thesis. The motivation of this research is to study the possibility of making enhancements in 3D pallet detection and localization system in an outdoor cargo and pallet handling site. At the time the task of my research was planned, classic methods were used for pallet detection purpose. The current research aims to open a new research track, which is focused on the deep learning object detection techniques, which has the potential to enhance the pallet detection performance using 3D LiDARs as the primary choice of a long range visual perception sensor that is not sensitive to outdoor lighting conditions. The main objectives of the this thesis include a) designing a simulated environment that is similar to an outdoor cargo handling site, b) designing a simulated robot and sensory ii system for object detection in Gazebo, c) writing a program that communicates with the simulated robot and the simulated world to receive the sensory data for further processing in robot operating system (ROS), d) writing a ground truth generation program to assist making a database of observations combined with precise 3D labelling, e) generation of a Kitti-structured database of many observations from the pallets and cargo in different placements and with their respective 3D bounding box labels, f) training a few deep learning structures with the generated database to analyze the learning performance, g) suggestion of the guidelines to enhance the detection results for the future work. In this thesis all the mentioned objectives were addressed, the results are reported, and the observations are discussed. Moreover, the database of pallet point cloud images is now accessible for download for any future works in this direction. I believe that that the outcome of this research plays a significant role in formation of a modern object detection technique that would replace the currently used classic method, or it would be used as a complementary work to enhance the pallet detection system.