Multi-camera tracking for smart urban mobility

The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior anal...

Full description

Saved in:
Bibliographic Details
Main Author: Do, Van Minh
Other Authors: Lam Siew Kei
Format: Thesis-Master by Research
Language:English
Published: Nanyang Technological University 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172263
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172263
record_format dspace
spelling sg-ntu-dr.10356-1722632024-01-04T06:32:51Z Multi-camera tracking for smart urban mobility Do, Van Minh Lam Siew Kei School of Computer Science and Engineering ASSKLam@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior analysis in commercial establishments. However, existing visual analytic methods adopted for MCT are computationally intensive, and often impractical for embedded systems. Additionally, challenges like camera viewpoint, distortion, low video resolutions, and varying environmental conditions can hinder MCT performance. In this study, we propose a novel algorithmic-hardware co-design methodology to accelerate and enhance the accuracy of MCT algorithms on resource-constrained platforms. We first introduce the algorithmic augmentations that facilitate adaptive region-of-interest (ROI) to restrict computations only on relevant scene regions, along with the use of sparse optical flow and linear estimation to improve tracking accuracy. To address computational limitations, we implement a multi-core processing strategy, with slower cores dynamically adjusting to computing needs and performing high-speed tracking to compensate for missing frames encountered by complex deep learning algorithms running on powerful cores. Experiments on an embedded device (Odroid N2+) using MOT15 and EPFL datasets demonstrate superior performance compared to baselines. In certain cases, the proposed method on Odroid N2+ even achieves comparable performance to high-end workstations. Next, we present a novel MCT approach that addresses key challenges in distributed camera networks. In order to enhance tracking accuracy and data association, our methodology incorporates the following tasks: change detection, optical flow augmentation, track similarity check, ROI-based track filtering, and adaptive travel time decay learning. Evaluations on AICITY, LAS, and Rafflescity datasets, on both workstation and Odroid N2+ platforms, demonstrate that our methodology can achieve significant improvements over existing methods. Finally, we conduct field trials in different locations in Singapore to validate the effectiveness of our methods in realistic urban mobility applications such as Smart Traffic Lights (STL) and Virtual Right of Way (VROW) for traffic flow management, and Infrastructure-to-Vehicle (I2V) communication for driving assistance. Our field trials successfully demonstrated the feasibility for deploying edge AI (Artificial Intelligence) devices in urban mobility applications. Master of Engineering 2023-12-04T08:26:21Z 2023-12-04T08:26:21Z 2023 Thesis-Master by Research Do, V. M. (2023). Multi-camera tracking for smart urban mobility. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/172263 https://hdl.handle.net/10356/172263 10.32657/10356/172263 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering::Computing methodologies
spellingShingle Engineering::Computer science and engineering::Computing methodologies
Do, Van Minh
Multi-camera tracking for smart urban mobility
description The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior analysis in commercial establishments. However, existing visual analytic methods adopted for MCT are computationally intensive, and often impractical for embedded systems. Additionally, challenges like camera viewpoint, distortion, low video resolutions, and varying environmental conditions can hinder MCT performance. In this study, we propose a novel algorithmic-hardware co-design methodology to accelerate and enhance the accuracy of MCT algorithms on resource-constrained platforms. We first introduce the algorithmic augmentations that facilitate adaptive region-of-interest (ROI) to restrict computations only on relevant scene regions, along with the use of sparse optical flow and linear estimation to improve tracking accuracy. To address computational limitations, we implement a multi-core processing strategy, with slower cores dynamically adjusting to computing needs and performing high-speed tracking to compensate for missing frames encountered by complex deep learning algorithms running on powerful cores. Experiments on an embedded device (Odroid N2+) using MOT15 and EPFL datasets demonstrate superior performance compared to baselines. In certain cases, the proposed method on Odroid N2+ even achieves comparable performance to high-end workstations. Next, we present a novel MCT approach that addresses key challenges in distributed camera networks. In order to enhance tracking accuracy and data association, our methodology incorporates the following tasks: change detection, optical flow augmentation, track similarity check, ROI-based track filtering, and adaptive travel time decay learning. Evaluations on AICITY, LAS, and Rafflescity datasets, on both workstation and Odroid N2+ platforms, demonstrate that our methodology can achieve significant improvements over existing methods. Finally, we conduct field trials in different locations in Singapore to validate the effectiveness of our methods in realistic urban mobility applications such as Smart Traffic Lights (STL) and Virtual Right of Way (VROW) for traffic flow management, and Infrastructure-to-Vehicle (I2V) communication for driving assistance. Our field trials successfully demonstrated the feasibility for deploying edge AI (Artificial Intelligence) devices in urban mobility applications.
author2 Lam Siew Kei
author_facet Lam Siew Kei
Do, Van Minh
format Thesis-Master by Research
author Do, Van Minh
author_sort Do, Van Minh
title Multi-camera tracking for smart urban mobility
title_short Multi-camera tracking for smart urban mobility
title_full Multi-camera tracking for smart urban mobility
title_fullStr Multi-camera tracking for smart urban mobility
title_full_unstemmed Multi-camera tracking for smart urban mobility
title_sort multi-camera tracking for smart urban mobility
publisher Nanyang Technological University
publishDate 2023
url https://hdl.handle.net/10356/172263
_version_ 1787590719380127744