Multi-camera tracking for smart urban mobility

• Main Author: Do, Van Minh
• Other Authors: Lam Siew Kei
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Computing methodologies
• Online Access: https://hdl.handle.net/10356/172263
• Institution: Nanyang Technological University

The demand for real-time multi-camera tracking (MCT) is rising with the increasing installation of roadside surveillance cameras to support urban mobility applications. A robust MCT system can aid traffic flow monitoring, investigation, driving assistance, law enforcement, and customer behavior analysis in commercial establishments. However, existing visual analytic methods adopted for MCT are computationally intensive, and often impractical for embedded systems. Additionally, challenges like camera viewpoint, distortion, low video resolutions, and varying environmental conditions can hinder MCT performance. In this study, we propose a novel algorithmic-hardware co-design methodology to accelerate and enhance the accuracy of MCT algorithms on resource-constrained platforms. We first introduce the algorithmic augmentations that facilitate adaptive region-of-interest (ROI) to restrict computations only on relevant scene regions, along with the use of sparse optical flow and linear estimation to improve tracking accuracy. To address computational limitations, we implement a multi-core processing strategy, with slower cores dynamically adjusting to computing needs and performing high-speed tracking to compensate for missing frames encountered by complex deep learning algorithms running on powerful cores. Experiments on an embedded device (Odroid N2+) using MOT15 and EPFL datasets demonstrate superior performance compared to baselines. In certain cases, the proposed method on Odroid N2+ even achieves comparable performance to high-end workstations. Next, we present a novel MCT approach that addresses key challenges in distributed camera networks. In order to enhance tracking accuracy and data association, our methodology incorporates the following tasks: change detection, optical flow augmentation, track similarity check, ROI-based track filtering, and adaptive travel time decay learning. Evaluations on AICITY, LAS, and Rafflescity datasets, on both workstation and Odroid N2+ platforms, demonstrate that our methodology can achieve significant improvements over existing methods. Finally, we conduct field trials in different locations in Singapore to validate the effectiveness of our methods in realistic urban mobility applications such as Smart Traffic Lights (STL) and Virtual Right of Way (VROW) for traffic flow management, and Infrastructure-to-Vehicle (I2V) communication for driving assistance. Our field trials successfully demonstrated the feasibility for deploying edge AI (Artificial Intelligence) devices in urban mobility applications.