Extrinsic calibration between thermal camera and mmWave radar for intelligent robots

• Main Author: Zhang, Shini
• Other Authors: Wang Dan Wei
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
• Online Access: https://hdl.handle.net/10356/159546
• Institution: Nanyang Technological University

LiDAR (Light Detection And Ranging) and RGB (Red-Green-Blue) camera perform well in general condition while 4D mmWave Radar (RAdio Detection And Ranging) and thermal camera do better in harsh environment. 4D Radar and thermal camera data fusion futher benefits intelligent robots to be used in various scenarios. There were limited literatures about these two sensors fusion as it was not long after the 4D Radar is pushed into market. Moreover, due to the heterogeneity of the two sensors, common feature is the main difficulty of the calibration. To solve these problems, 4DRadar2ThermalCalib is proposed. The three main contributions of this dissertation are: 1) A systematic intrinsic and extrinsic calibration method, 4DRadar2ThermalCalib, for a 4D mmWave Radar and a thermal camera is proposed. 2) A novel calibration target, sphericaltrihedral, is designed to provide the common features between a 4D mmWave Radar and a thermal camera. 3) Sphere center features in both Radar point cloud data and thermal image data are detected automatically. The results of the extrinsic calibration is obtained by minimizing the re-projection error. Both quantitative and qualitative analysis are implemented to prove that the 4DRadar2ThermalCalib method performs well in real environment. The overall re-projection error is 1.88 pixels.