Demonstrating canvas-based processing of multiple camera streams at the edge

• Main Authors: GOKARN, Ila, SABBELLA, Hemanth, HU, Yigong, ABDELZAHER, Tarek, MISRA, Archan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Canvas-based Processing; Edge Computation; Multi-Camera Systems; Graphics and Human Computer Interfaces; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/9224; https://ink.library.smu.edu.sg/context/sis_research/article/10230/viewcontent/COMSNETS2024_Ila_cameraready.pdf
• Institution: Singapore Management University

We demonstrate criticality-aware canvas-based processing of multiple concurrent camera streams at the resource constrained edge to show substantial improvement in the accuracy-throughput trade-off. The proposed system focuses the available computation resources on select Regions of Interest (RoI) across all the camera streams by (i) extracting RoI from the input camera stream (ii) 2D bin packing the RoI on a canvas frame and (iii) batching and inferring upon these constructed composite canvas frames with a YOLOv5 object detection model. Our experiments show that such canvas-based processing can (i) sustain real-time processing throughput of 23 FPS per camera across 6 concurrent input camera streams (cumulatively 138 FPS) on a single NVIDIA Jetson TX2 representing a 475% increase in throughput, with (ii) negligible loss in accuracy as compared to a First Come First Serve (FCFS) baseline running full frame detections on the input camera streams.