Multi-camera tracking for smart urban mobility
The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior anal...
Saved in:
Main Author: | |
---|---|
Other Authors: | |
Format: | Thesis-Master by Research |
Language: | English |
Published: |
Nanyang Technological University
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/172263 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-172263 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1722632024-01-04T06:32:51Z Multi-camera tracking for smart urban mobility Do, Van Minh Lam Siew Kei School of Computer Science and Engineering ASSKLam@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior analysis in commercial establishments. However, existing visual analytic methods adopted for MCT are computationally intensive, and often impractical for embedded systems. Additionally, challenges like camera viewpoint, distortion, low video resolutions, and varying environmental conditions can hinder MCT performance. In this study, we propose a novel algorithmic-hardware co-design methodology to accelerate and enhance the accuracy of MCT algorithms on resource-constrained platforms. We first introduce the algorithmic augmentations that facilitate adaptive region-of-interest (ROI) to restrict computations only on relevant scene regions, along with the use of sparse optical flow and linear estimation to improve tracking accuracy. To address computational limitations, we implement a multi-core processing strategy, with slower cores dynamically adjusting to computing needs and performing high-speed tracking to compensate for missing frames encountered by complex deep learning algorithms running on powerful cores. Experiments on an embedded device (Odroid N2+) using MOT15 and EPFL datasets demonstrate superior performance compared to baselines. In certain cases, the proposed method on Odroid N2+ even achieves comparable performance to high-end workstations. Next, we present a novel MCT approach that addresses key challenges in distributed camera networks. In order to enhance tracking accuracy and data association, our methodology incorporates the following tasks: change detection, optical flow augmentation, track similarity check, ROI-based track filtering, and adaptive travel time decay learning. Evaluations on AICITY, LAS, and Rafflescity datasets, on both workstation and Odroid N2+ platforms, demonstrate that our methodology can achieve significant improvements over existing methods. Finally, we conduct field trials in different locations in Singapore to validate the effectiveness of our methods in realistic urban mobility applications such as Smart Traffic Lights (STL) and Virtual Right of Way (VROW) for traffic flow management, and Infrastructure-to-Vehicle (I2V) communication for driving assistance. Our field trials successfully demonstrated the feasibility for deploying edge AI (Artificial Intelligence) devices in urban mobility applications. Master of Engineering 2023-12-04T08:26:21Z 2023-12-04T08:26:21Z 2023 Thesis-Master by Research Do, V. M. (2023). Multi-camera tracking for smart urban mobility. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/172263 https://hdl.handle.net/10356/172263 10.32657/10356/172263 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering::Computing methodologies |
spellingShingle |
Engineering::Computer science and engineering::Computing methodologies Do, Van Minh Multi-camera tracking for smart urban mobility |
description |
The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior analysis in commercial establishments. However, existing visual analytic methods adopted for MCT are computationally intensive, and often impractical for embedded systems. Additionally, challenges like camera viewpoint, distortion, low video resolutions, and varying environmental conditions can hinder MCT performance. In this study, we propose a novel algorithmic-hardware co-design methodology to accelerate and enhance the accuracy of MCT algorithms on resource-constrained platforms.
We first introduce the algorithmic augmentations that facilitate adaptive region-of-interest (ROI) to restrict computations only on relevant scene regions, along with the use of sparse optical flow and linear estimation to improve tracking accuracy. To address computational limitations, we implement a multi-core processing strategy, with slower cores dynamically adjusting to computing needs and performing high-speed tracking to compensate for missing frames encountered by complex deep learning algorithms running on powerful cores. Experiments on an embedded device (Odroid N2+) using MOT15 and EPFL datasets demonstrate superior performance compared to baselines. In certain cases, the proposed method on Odroid N2+ even achieves comparable performance to high-end workstations.
Next, we present a novel MCT approach that addresses key challenges in distributed camera networks. In order to enhance tracking accuracy and data association, our methodology incorporates the following tasks: change detection, optical flow augmentation, track similarity check, ROI-based track filtering, and adaptive travel time decay learning. Evaluations on AICITY, LAS, and Rafflescity datasets, on both workstation and Odroid N2+ platforms, demonstrate that our methodology can achieve significant improvements over existing methods.
Finally, we conduct field trials in different locations in Singapore to validate the effectiveness of our methods in realistic urban mobility applications such as Smart Traffic Lights (STL) and Virtual Right of Way (VROW) for traffic flow management, and Infrastructure-to-Vehicle (I2V) communication for driving assistance. Our field trials successfully demonstrated the feasibility for deploying edge AI (Artificial Intelligence) devices in urban mobility applications. |
author2 |
Lam Siew Kei |
author_facet |
Lam Siew Kei Do, Van Minh |
format |
Thesis-Master by Research |
author |
Do, Van Minh |
author_sort |
Do, Van Minh |
title |
Multi-camera tracking for smart urban mobility |
title_short |
Multi-camera tracking for smart urban mobility |
title_full |
Multi-camera tracking for smart urban mobility |
title_fullStr |
Multi-camera tracking for smart urban mobility |
title_full_unstemmed |
Multi-camera tracking for smart urban mobility |
title_sort |
multi-camera tracking for smart urban mobility |
publisher |
Nanyang Technological University |
publishDate |
2023 |
url |
https://hdl.handle.net/10356/172263 |
_version_ |
1787590719380127744 |