Real-time pedestrian detection based on multi-modal sensor fusion
Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable prog...
Saved in:
Main Author: | |
---|---|
Other Authors: | |
Format: | Thesis-Master by Coursework |
Language: | English |
Published: |
Nanyang Technological University
2022
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/155526 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-155526 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1555262023-07-04T17:41:35Z Real-time pedestrian detection based on multi-modal sensor fusion Wang, Ziyue Wang Dan Wei School of Electrical and Electronic Engineering EDWWANG@ntu.edu.sg Engineering::Electrical and electronic engineering Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable progress in the field of pedestrian detection. Fast and accurate detection algorithm can supply more information for the driving assistant system. However, the overall detection performance is still limited to the characteristics of the sensor: detection based on RGB-D camera is limited to the performance of depth camera, while the detection based on complete point cloud data consumes unacceptable computing resources in case of real-time pedestrians detection. In this project, a synthesized survey is firstly conducted to find out the existing detection models. Several state-of-the-art algorithms are then evaluated and compared through Carla simulator, including algorithms which detected only based on camera data, that only based on LiDAR data and that based on both camera data and LiDAR data. Also, several tracking algorithms are compared to pass the information of detected pedestrian to fusion processor. Finally, a real-time pedestrian detection system is established to reach the goal of quick and accurate detection. The result indicates that our detection system has robust and satisfying performance on simulation scene. In addition, the system is designed to be flexible. Users can modify the configuration file to easily selected the needed plugins such as enable LiDAR detection and enable frustum detection. The content of this report includes the introduction of several state-of-the-art detection algorithms, the design of the system and the validation on simulator. Moreover, there is a discussion which focuses on the performance of each detection algorithm and the overall simulation result. Master of Science (Computer Control and Automation) 2022-03-02T03:07:27Z 2022-03-02T03:07:27Z 2022 Thesis-Master by Coursework Wang, Z. (2022). Real-time pedestrian detection based on multi-modal sensor fusion. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/155526 https://hdl.handle.net/10356/155526 en application/pdf Nanyang Technological University |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Electrical and electronic engineering |
spellingShingle |
Engineering::Electrical and electronic engineering Wang, Ziyue Real-time pedestrian detection based on multi-modal sensor fusion |
description |
Pedestrian detection in driving assistant system refers to obtain the 3-d coordinate of the pedestrians nearby through the information from the sensors such as RGB camera, depth camera, LiDAR or RADAR. The success implement of deep learning approaches in Computer Vision has spurred considerable progress in the field of pedestrian detection. Fast and accurate detection algorithm can supply more information for the driving assistant system. However, the overall detection performance is still limited to the characteristics of the sensor: detection based on RGB-D camera is limited to the performance of depth camera, while the detection based on complete point cloud data consumes unacceptable computing resources in case of real-time pedestrians detection.
In this project, a synthesized survey is firstly conducted to find out the existing detection models. Several state-of-the-art algorithms are then evaluated and compared through Carla simulator, including algorithms which detected only based on camera data, that only based on LiDAR data and that based on both camera data and LiDAR data. Also, several tracking algorithms are compared to pass the information of detected pedestrian to fusion processor. Finally, a real-time pedestrian detection system is established to reach the goal of quick and accurate detection.
The result indicates that our detection system has robust and satisfying performance on simulation scene. In addition, the system is designed to be flexible. Users can modify the configuration file to easily selected the needed plugins such as enable LiDAR detection and enable frustum detection.
The content of this report includes the introduction of several state-of-the-art detection algorithms, the design of the system and the validation on simulator. Moreover, there is a discussion which focuses on the performance of each detection algorithm and the overall simulation result. |
author2 |
Wang Dan Wei |
author_facet |
Wang Dan Wei Wang, Ziyue |
format |
Thesis-Master by Coursework |
author |
Wang, Ziyue |
author_sort |
Wang, Ziyue |
title |
Real-time pedestrian detection based on multi-modal sensor fusion |
title_short |
Real-time pedestrian detection based on multi-modal sensor fusion |
title_full |
Real-time pedestrian detection based on multi-modal sensor fusion |
title_fullStr |
Real-time pedestrian detection based on multi-modal sensor fusion |
title_full_unstemmed |
Real-time pedestrian detection based on multi-modal sensor fusion |
title_sort |
real-time pedestrian detection based on multi-modal sensor fusion |
publisher |
Nanyang Technological University |
publishDate |
2022 |
url |
https://hdl.handle.net/10356/155526 |
_version_ |
1772827749082202112 |