Implementation of a panoramic image stitcher for a deep tunnel robotic platform

• Main Author: Kee, Da Wei
• Other Authors: Seet Gim Lee, Gerald
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Mechanical engineering::Mechatronics
• Online Access: https://hdl.handle.net/10356/150825
• Institution: Nanyang Technological University

Underground deep tunnels were constructed for use in transportation and collection of used water around Singapore. Due to the hazardous working environment inside the tunnels, a deep tunnel robotic platform for performing inspection of the tunnel linings was developed by a collaborative effort between Nanyang Technological University (NTU) and Public Utilities Board (PUB). The robotic platform operated by a handler above-ground, is equipped with various cameras in providing visual information to the operator for inspection of the tunnel linings, and the navigation of the robotic platform. The visual information is also stored in the data server in the form of videos for use in playback and post-image processing. Through the image stitching technique, a layout map of the deep tunnels could be recreated in the form of a single continuous flat image of the deep tunnel wall. The author evaluated different image stitching algorithms, implemented in various robotic vision platforms, as well as different image processing software and libraries used for computer vision, to design and implement an optimised C++ image stitching algorithm for the deep tunnel robotic platform. The image stitching algorithm can create a stitched map of the deep tunnel linings from the recorded video taken by the robotic platform after deployment. The implemented image stitching algorithm would first remove any redundant images (frames) from the recorded video through the use of motion estimation, to hasten the image stitching process. The resulting frames would undergo image warping to remove distortion present in the image due to the curvature of the tunnels and the lens used in the camera. The overlapping regions between the frames were matched and stitched together to create the stitched map of deep tunnel linings. A special image straightening process was implemented alongside the stitching algorithm to ensure the resulting stitched map remained horizontal. The resulting stitched map was shown and discussed to determine the accuracy and effectiveness of the implemented image stitching algorithms. In addition, further improvements to the image stitching algorithm and future works, which include using the resulting stitched map for anomalies detection, were also discussed.