Sensor fusion for object detection under adverse weather

Autonomous vehicles (AVs) have rapidly emerged as one of the most rapidly advancing technologies in the realm of transportation. The innovative strides in AV technology, from self-driving cars to autonomous delivery drones, have captured widespread attention and are poised to reshape the future o...

Full description

Saved in:
Bibliographic Details
Main Author: Soh, Brandon Jian Zheng
Other Authors: Soong Boon Hee
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172746
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172746
record_format dspace
spelling sg-ntu-dr.10356-1727462023-12-22T15:43:55Z Sensor fusion for object detection under adverse weather Soh, Brandon Jian Zheng Soong Boon Hee School of Electrical and Electronic Engineering EBHSOONG@ntu.edu.sg Engineering::Computer science and engineering Engineering::Electrical and electronic engineering Autonomous vehicles (AVs) have rapidly emerged as one of the most rapidly advancing technologies in the realm of transportation. The innovative strides in AV technology, from self-driving cars to autonomous delivery drones, have captured widespread attention and are poised to reshape the future of mobility. Traditional Autonomous Vehicle (AV) camerabased object detectors fail in complicated scenarios where the lighting and weather conditions are not ideal. In recent years, there has been a rise in the use of deep learning methods relying on LiDARs and Radars, given their long history of achieving state of art performance in different types of applications. Despite the rapid development in deep neural networks, object detection is still a persistent challenge as some sensors experience poor perception under adverse weather. Severe weather conditions can still impede LiDAR’s performance despite its excellent capabilities. Snow and water particles make the targets detected by the LiDAR sensor become partially occluded. To address this redundancy, a RADAR sensor is introduced to compensate for LiDAR’s shortcomings. In this project, a sensor fusion approach is proposed to optimize the object detection’s performance in rainy, snowy, or foggy weather. In addition, a comparison of the performance of the object detectors using only one of the proposed sensors will also be performed. Adding on, catastrophic forgetting is a baffling phenomenon that can be observed in neural networks. This happens when a trained neural network has the tendency of losing its ability to make predictions based on its previous training when it is finetuned with new data. Therefore, an investigation will be conducted on the impact of catastrophic forgetting in adverse weather conditions for both LiDAR and RADAR sensors. A comparative assessment will be made to formulate probable resolution. Bachelor of Engineering (Computer Engineering) 2023-12-19T07:18:30Z 2023-12-19T07:18:30Z 2023 Final Year Project (FYP) Soh, B. J. Z. (2023). Sensor fusion for object detection under adverse weather. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/172746 https://hdl.handle.net/10356/172746 en application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Engineering::Electrical and electronic engineering
spellingShingle Engineering::Computer science and engineering
Engineering::Electrical and electronic engineering
Soh, Brandon Jian Zheng
Sensor fusion for object detection under adverse weather
description Autonomous vehicles (AVs) have rapidly emerged as one of the most rapidly advancing technologies in the realm of transportation. The innovative strides in AV technology, from self-driving cars to autonomous delivery drones, have captured widespread attention and are poised to reshape the future of mobility. Traditional Autonomous Vehicle (AV) camerabased object detectors fail in complicated scenarios where the lighting and weather conditions are not ideal. In recent years, there has been a rise in the use of deep learning methods relying on LiDARs and Radars, given their long history of achieving state of art performance in different types of applications. Despite the rapid development in deep neural networks, object detection is still a persistent challenge as some sensors experience poor perception under adverse weather. Severe weather conditions can still impede LiDAR’s performance despite its excellent capabilities. Snow and water particles make the targets detected by the LiDAR sensor become partially occluded. To address this redundancy, a RADAR sensor is introduced to compensate for LiDAR’s shortcomings. In this project, a sensor fusion approach is proposed to optimize the object detection’s performance in rainy, snowy, or foggy weather. In addition, a comparison of the performance of the object detectors using only one of the proposed sensors will also be performed. Adding on, catastrophic forgetting is a baffling phenomenon that can be observed in neural networks. This happens when a trained neural network has the tendency of losing its ability to make predictions based on its previous training when it is finetuned with new data. Therefore, an investigation will be conducted on the impact of catastrophic forgetting in adverse weather conditions for both LiDAR and RADAR sensors. A comparative assessment will be made to formulate probable resolution.
author2 Soong Boon Hee
author_facet Soong Boon Hee
Soh, Brandon Jian Zheng
format Final Year Project
author Soh, Brandon Jian Zheng
author_sort Soh, Brandon Jian Zheng
title Sensor fusion for object detection under adverse weather
title_short Sensor fusion for object detection under adverse weather
title_full Sensor fusion for object detection under adverse weather
title_fullStr Sensor fusion for object detection under adverse weather
title_full_unstemmed Sensor fusion for object detection under adverse weather
title_sort sensor fusion for object detection under adverse weather
publisher Nanyang Technological University
publishDate 2023
url https://hdl.handle.net/10356/172746
_version_ 1787136790224699392