EDLAB : a benchmark for edge deep learning accelerators

• Main Authors: Kong, Hao, Huai, Shuo, Liu, Di, Zhang, Lei, Chen, Hui, Zhu, Shien, Li, Shiqing, Liu, Weichen, Rastogi, Manu, Subramaniam, Ravi, Athreya, Madhu, Lewis, M. Anthony
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2022
• Subjects: Engineering::Computer science and engineering; Benchmark; Deep Learning; Edge Accelerator; Deployment
• Online Access: https://hdl.handle.net/10356/155807
• Institution: Nanyang Technological University

A new trend tends to deploy deep learning algorithms to edge environments to mitigate privacy and latency issues from cloud computing. Diverse edge deep learning accelerators are devised to speed up the inference of deep learning algorithms on edge devices. Various edge deep learning accelerators feature different characteristics in terms of power and performance, which make it a very challenging task to efficiently and uniformly compare different accelerators. In this paper, we introduce EDLAB, an end-to-end benchmark, to evaluate the overall performance of edge deep learning accelerators. EDLAB consists of state-of-the-art deep learning models, a unified workload preprocessing and deployment framework, as well as a collection of comprehensive metrics. In addition, we propose parameterized models to model the hardware performance bound so that EDLAB can identify the hardware potentials and the hardware utilization of different deep learning applications. Finally, we employ EDLAB to benchmark three edge deep learning accelerators and analyze the benchmarking results. From the analysis we obtain some insightful observations that can guide the design of efficient deep learning applications.