Real-time pedestrian detection based on multi-modal sensor fusion

Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable prog...

Full description

Saved in:
Bibliographic Details
Main Author: Wang, Ziyue
Other Authors: Wang Dan Wei
Format: Thesis-Master by Coursework
Language:English
Published: Nanyang Technological University 2022
Subjects:
Online Access:https://hdl.handle.net/10356/155526
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-155526
record_format dspace
spelling sg-ntu-dr.10356-1555262023-07-04T17:41:35Z Real-time pedestrian detection based on multi-modal sensor fusion Wang, Ziyue Wang Dan Wei School of Electrical and Electronic Engineering EDWWANG@ntu.edu.sg Engineering::Electrical and electronic engineering Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable progress in the field of pedestrian detection. Fast and accurate detection algorithm can supply more information for the driving assistant system. However, the overall detection performance is still limited to the characteristics of the sensor: detection based on RGB-D camera is limited to the performance of depth camera, while the detection based on complete point cloud data consumes unacceptable computing resources in case of real-time pedestrians detection. In this project, a synthesized survey is firstly conducted to find out the existing detection models. Several state-of-the-art algorithms are then evaluated and compared through Carla simulator, including algorithms which detected only based on camera data, that only based on LiDAR data and that based on both camera data and LiDAR data. Also, several tracking algorithms are compared to pass the information of detected pedestrian to fusion processor. Finally, a real-time pedestrian detection system is established to reach the goal of quick and accurate detection. The result indicates that our detection system has robust and satisfying performance on simulation scene. In addition, the system is designed to be flexible. Users can modify the configuration file to easily selected the needed plugins such as enable LiDAR detection and enable frustum detection. The content of this report includes the introduction of several state-of-the-art detection algorithms, the design of the system and the validation on simulator. Moreover, there is a discussion which focuses on the performance of each detection algorithm and the overall simulation result. Master of Science (Computer Control and Automation) 2022-03-02T03:07:27Z 2022-03-02T03:07:27Z 2022 Thesis-Master by Coursework Wang, Z. (2022). Real-time pedestrian detection based on multi-modal sensor fusion. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/155526 https://hdl.handle.net/10356/155526 en application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
spellingShingle Engineering::Electrical and electronic engineering
Wang, Ziyue
Real-time pedestrian detection based on multi-modal sensor fusion
description Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable progress in the field of pedestrian detection. Fast and accurate detection algorithm can supply more information for the driving assistant system. However, the overall detection performance is still limited to the characteristics of the sensor: detection based on RGB-D camera is limited to the performance of depth camera, while the detection based on complete point cloud data consumes unacceptable computing resources in case of real-time pedestrians detection. In this project, a synthesized survey is firstly conducted to find out the existing detection models. Several state-of-the-art algorithms are then evaluated and compared through Carla simulator, including algorithms which detected only based on camera data, that only based on LiDAR data and that based on both camera data and LiDAR data. Also, several tracking algorithms are compared to pass the information of detected pedestrian to fusion processor. Finally, a real-time pedestrian detection system is established to reach the goal of quick and accurate detection. The result indicates that our detection system has robust and satisfying performance on simulation scene. In addition, the system is designed to be flexible. Users can modify the configuration file to easily selected the needed plugins such as enable LiDAR detection and enable frustum detection. The content of this report includes the introduction of several state-of-the-art detection algorithms, the design of the system and the validation on simulator. Moreover, there is a discussion which focuses on the performance of each detection algorithm and the overall simulation result.
author2 Wang Dan Wei
author_facet Wang Dan Wei
Wang, Ziyue
format Thesis-Master by Coursework
author Wang, Ziyue
author_sort Wang, Ziyue
title Real-time pedestrian detection based on multi-modal sensor fusion
title_short Real-time pedestrian detection based on multi-modal sensor fusion
title_full Real-time pedestrian detection based on multi-modal sensor fusion
title_fullStr Real-time pedestrian detection based on multi-modal sensor fusion
title_full_unstemmed Real-time pedestrian detection based on multi-modal sensor fusion
title_sort real-time pedestrian detection based on multi-modal sensor fusion
publisher Nanyang Technological University
publishDate 2022
url https://hdl.handle.net/10356/155526
_version_ 1772827749082202112