Visual inertial odometry and lidar inertial odometry for mobile robot

• Main Author: Henawy, John Farid Nasry
• Other Authors: Seet Gim Lee, Gerald
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Mechanical engineering
• Online Access: https://hdl.handle.net/10356/146364
• Institution: Nanyang Technological University

Municipal infrastructure inspection and maintenance are vital to keep the integrity and safe operation of large-scale infrastructure such as pipelines, bridges and tunnels. For example, regular inspection and assessment of tunnels are essential to prevent any possible leakage or worse, collapse. Inspection of deep sewerage tunnels is even more challenging because of their complex environment where such tunnels are not only difficult to reach, they may also have dangerous bacteria and microorganisms and the sewer gas would also have mixture of flammable and toxic gases such as hydrogen sulphide, methane, ammonia and sulphur dioxide. In Singapore, deep tunnel sewerage system (DTSS) is 6 m in diameter and located about 20 to 50 m underground. Recently, quadcopters are widely used for inspection because of their ability to do vertical take-off and landing with high stability, reach to great height as well as ease of use and low cost. However, quadcopter localization in deep sewerage tunnel is very challenging as the environment there is dark, wet, featureless and structureless. The darkness and featureless environment causes significant visual degradation, while structureless and wet environment causes lidar degradation. Thus, multiple sensors are required for proper robot localization to mitigate such sensor degradation. Moreover, employing an inertial measurement unit (IMU) as an additional sensor can dramatically improve both reliability and accuracy of visual and lidar odometry. Therefore, data fusion paradigm has become the major focus for the visual inertial odometry (VIO) and lidar inertial odometry (LIO) algorithms. Among these, the filtering data fusion paradigm is a common fusion technique because it is fast and efficient for online estimation. However, it produces cumulative linearization error which will cause gradual deterioration in the accuracy. In this thesis, a data fusion paradigm based on nonlinear optimization called a full smoothing paradigm has been proposed as a better alternative to the filtering paradigm. The full smoothing paradigm takes into consideration the full history of the previous states without marginalization. Therefore, it is more accurate than the filtering paradigms. However, high IMU sampling rate represents a great challenge for online estimation. This is because the full state history grows rapidly making online estimation not feasible. Motion preintegration overcomes this problem whereby the IMU measurements can be integrated into a single equivalent value between two visual or lidar keyframes. This makes the full smoothing paradigm simpler and the nonlinear optimization process can be done at a lower frequency. In this work, a novel IMU motion integration model was proposed for the continuous-time of IMU kinematics which are modelled using a switched linear system. The novel IMU model has a closed-form solution. Based on that, a novel IMU factor was formulated as a closed-form discrete factor to compute the mean measurements, covariance matrix and Jacobians. As such, the proposed model is more accurate and more efficient for online state estimation than the state-of-the-art. To evaluate the proposed IMU factor, it is integrated with the VIO and LIO frameworks as tightly coupled joint-optimization. The proposed VIO has been evaluated using simulated and real-world datasets. In addition, indoor experiments have been done to test its effectiveness. Results obtained show that the proposed framework outperforms the state-of-the-art approach by up to 22 % and 38 % on real datasets and indoor experiments, respectively. A novel nonlinear de-skewing is then proposed to correct the motion distortion between all scanned point in LIO. The novel IMU motion integration model is further used to correct the distorted points. In addition, the novel IMU factor is used with lidar factor in joint-optimization to obtain high state estimation. Results obtained show that the proposed model outperforms the state-of-the-art by up to 21 %.