Vision-based road lane and curb detection for unmanned ground vehicles in urban scenarios

• Main Author: Zhou, Hui
• Other Authors: Xie Lihua
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/172769
• Institution: Nanyang Technological University

This thesis presents a study on the vision-based lane and curb detection for Unmanned Ground Vehicles on urban scenarios. The perception of road lane markings is a significant component of unmanned ground vehicles for lane-level localization, lane keeping and unsteady driving alerting. Curb is also an important road feature to assist autonomous driving when lane markings can not be found. It can provide road boundary information and also a potential risk of severe tire damage which should be aware of to improve traffic efficiency. For the above two mentioned research topics, the requirement of real-time processing speed on a low power processor is a main limitation for the algorithms to be applied. The first challenge addressed of lane detection is the sophisticated real driving situations (e.g. emerging, ending, spitting and merging of lane markings, heavily curved lanes, zig-zag lanes, on/off ramps and disturbance of other road writings). To describe a wider range of lane structures, the key contribution is to associate lane feature clusters through energy minimization without any assumption of lane and road model. To facilitate lane tracking, we introduce the lane appearance matching and propose probabilistic decision making combing prior lane state with lane feature cluster candidate measurements. A multi-lane detection and tracking framework is presented which exploits the temporal coherence of lane appearance to improve the smooth detection and robustness to outliers. The proposed framework has been tested in the real world with live data on public roads in real time. The second part of the study is on curb detection for the safe navigation of the robot. Curb detection is more challenging than lane detection because curb feature is not constant regarding its texture, geometry, or gradient information. The key contribution in this section is to propose to use 3D point observations obtained from camera calibration files and disparity map which can be generated with stereo camera. The proposed algorithm consists of two parts. The first part is a conversion from a disparity map to a Digital Elevation Map (DEM) structure. The remaining part is the detection of the road surfaces and two curbs on the DEM. The curb is defined as the horizontal separation of the road surface and its adjacent surface like sidewalks, hence there are obvious height discontinuities which are utilized to check the presence of curbs. The proposed framework also has been tested in the real world with live data on public roads in real time. The third contribution of the thesis is to propose a unified network to incorporate these two tasks together by taking advantage of the powerful feature learning ability brought by deep convolutional neural networks. Existing vision-based researches treat lane detection and curb detection separately due to the nature of curb detection problem relying on 3D features, which is not efficient when driving in real world. The unified network accurately differentiates various lane and curb instances with tiny gaps or complex spatial relationships which are considered as the biggest challenge in real driving situations. To achieve this, the network is specially designed to guide lane and curb instances to learnable kernel through pixel grouping. This learnable kernel is capable to handle any number of arbitraryshaped lanes and curbs no matter which angle the vehicle is heading at. In the end, the presented approach is evaluated on two datasets (BDD100K and self-collected dataset) scoring 32 FPS processing speed. Results are very encouraging with more than 98% F1 measure for both lane and curb detection on the self-collected dataset.