Lane-aware image enhancement for lane detection in rain (part B - communications)

• Main Author: Tian, Zichen
• Other Authors: Soong Boon Hee
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Pattern recognition; Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/150156
• Institution: Nanyang Technological University

In recent years, auto-driving is becoming a hot topic. Auto-driving cars utilize in-vehicle cameras to capture the surrounding environment images and use algorithms to extract useful information from images. One of the most critical issues to be solved in auto-driving is lane detection and road marking recognition. By implementing lane and road marking detection algorithms, surrounding traffic symbols can be recognized and used to help the human driver avoid accidents. Two central problems remain in such process: 1) the curved road is not easy to detect; 2) the rainy condition may distort the image captured by cameras, thus hard to recognize. In this dissertation, a network structure RVPGNet, based on previous work VPGNet, is defined to address above mentioned problems. Novel ways of combining the information and efficiently employing the abstract training data are proposed. Inspired by VPGNet, the RVPGNet employs multi-task to do lane detection and road marking classification tasks simultaneously, and it also utilizes the vanishing point to guide the lane prediction. Besides, RVPGNet features in four innovative combination layers and algorithms: 1) To save computational resources, a new information combination layer, called the 4-tiling layer, was proposed and applied; Two new feeding schemes, called 2) N-map layer and 3) 2-D Gaussian feeding layer, were designed to utilize the vanishing point better and avoid training being trapped at the saddle point; 4) The network is implemented in Caffe framework and currently under construction in PyTorch framework. The experimental results are significant. In the Caffe implementation, a 97:19% accuracy is achieved in multi-label classification. In the test of rainy conditions, the network achieves as high as 93:35% to 99:73% F1 score in the blurry and low-brightness images. Currently, we are transplanting the network from Caffe to the state-of-the-art PyTorch framework. The overall structure has been constructed and debugging on test metric and backpropagation is in progress. Based on this dissertation’s work, many valuable improvements can be made in the future. 1) The PyTorch implementation’s main structure is finished, so future research only needs to refine the backpropagation, implement the 2-D Gaussian in Torch tensors. Besides, it is worthy to refine the selection of the initialization function. 2) A loss function measuring the offset of Vanishing Point and ground truth can be constructed; 3) Re-scale the network from low-definition to high resolution for pixel-to-pixel classification. All the diagrams are drawn in vector graph format, so readers can zoom in to check the details.