<i>MRIM:</i> Lightweight saliency-based mixed-resolution imaging for low-power pervasive vision

• Main Authors: WU, Jiyan, SUBASHARAN, Vithurson, TRAN, Minh Anh Tuan, GAMLATH, Kasun Pramuditha, MISRA, Archan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Mixed resolution; pervasive vision tasks; energy consumption; Graphics and Human Computer Interfaces; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/8488; https://ink.library.smu.edu.sg/context/sis_research/article/9491/viewcontent/MRIM_sv.pdf
• Institution: Singapore Management University

While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce MRIM, a simple but effective framework to tackle this tradeoff. Under MRIM, the vision sensor platform first executes a lightweight preprocessing step to determine the saliency of different sub-regions within a single captured image frame, and then performs a saliency-aware non-uniform downscaling of individual sub-regions to produce a “mixed-resolution” image. We describe two novel low-complexity algorithms that the sensor platform can use to quickly compute suitable resolution choices for different regions under different energy/accuracy constraints. Experimental studies, involving object detection tasks evaluated traces from two benchmark urban monitoring datasets as well as a prototype Raspberry Pi-based MRIM implementation, demonstrate MRIM’s efficacy: even with an unoptimized embedded platform, MRIM can provide system energy conservation of 35+ % (~80% in high accuracy regimes) or increase task accuracy by 8+ %, over conventional baselines of uniform resolution downscaling or image encoding, while supporting high throughput. On a low power ESP32 vision board, MRIM continues to provide 60+% energy savings over uniform downscaling while maintaining high detection accuracy. We further introduce an automated data-driven technique for determining a close-to-optimal number of MRIM sub-regions (for differential resolution adjustment), across different deployment conditions. We also show the generalized use of MRIM by considering an additional license plate recognition (LPR) task: while alternative approaches suffer 35%–40% loss in accuracy, MRIM suffers only a modest recognition loss of ~10% even when the transmission data is reduced by over 50%.