Criticality aware canvas-based visual perception at the edge

• Main Author: GOKARN, Ila
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Edge AI; Machine Perception; Canvas-based Processing; Artificial Intelligence and Robotics; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/9231; https://ink.library.smu.edu.sg/context/sis_research/article/10231/viewcontent/3643832.3661386.pdf
• Institution: Singapore Management University

Efficient and effective machine perception remains a formidable challenge in sustaining high fidelity and high throughput of perception tasks on affordable edge devices. This is especially due to the continuing increase in resolution of sensor streams (e.g., video input streams generated by 4K/8K cameras and neuromorphic event cameras that produce ≥ 10 MEvents/second) and computational complexity of Deep Neural Network (DNN) models, which overwhelms edge platforms, adversely impacting machine perception efficiency. Given the insufficiency of the available computation resources, a question then arises on whether selected regions/components of the perception task can be prioritized (and executed preferentially) to achieve highest task fidelity while adhering to the resource budget. This extended abstract explores the paradigm of Canvas-based Processing and criticality-awareness in the context of multi-sensor machine perception pipelines on resource-constrained platforms, in guiding perception pipelines and systems on “what" to pay attention to in the sensing field and “when", to maximize overall perception fidelity under computational constraints and moderate the processing throughput-vs-accuracy trade-off.