Design of robust sensor suite by fusing 3D LiDAR, RGB camera, thermal camera and IMU

• Main Author: Chandrasekarapuram Krishnan Karthick
• Other Authors: Wang Dan Wei
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
• Online Access: https://hdl.handle.net/10356/156986
• Institution: Nanyang Technological University

Nowadays, more and more robots are mounted with multiple heterogeneous sensors to ensure robust perception in all weathers. Each sensor has advantages and disadvantages. For example, LiDAR can perceive the depth information of the environment, whereas RGB camera can perceive the color information. Likewise, the RGB camera cannot work in darkness, whereas the Thermal camera can work even in darkness. Similarly, IMU operates at a higher frequency while other sensors operate at a lower frequency. Hence, by combining the multiple sensors perception can be improved. However, currently, the sensors are mounted independently one by one, and the calibration between the sensors is laborious and time-consuming. It is expected to make an all-in-one and compact sensor suite, which already includes accurate intrinsic and extrinsic calibration parameters. Such an all-in-one sensor suite can save time for related applications. This dissertation is focused on designing such a sensor suite. It mainly consists of three parts: 1) Sensor suite design. A compact sensor suite is designed and fabricated. It consists of a 3D LiDAR, an RGB camera, a thermal camera, and an IMU. Two versions are designed. Autodesk Fusion360 is used to design the hardware. 3D printing is utilized to fabricate the hardware. 2) Intrinsic and extrinsic calibration of and between the four sensors. Intrinsic calibration is performed for RGB camera, Thermal camera, and IMU. Extrinsic calibration is performed for 3D LiDAR-RGB camera, RGB-Thermal camera, 3D LiDAR-Thermal camera, IMU–RGB camera. Both quantitative and qualitative analyses are performed to analyze the calibration error in the experiments. 3) One application of the sensor suite – color 3D mapping. With the sensor suite and calibration parameters, a color 3D map is constructed by fusing multi-source information from the sensors. Experiments demonstrate the effectiveness of color 3D mapping.