You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms

Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios,...

Full description

Saved in:
Bibliographic Details
Main Authors: Luo, Xiangzhong, Liu, Di, Kong, Hao, Huai, Shuo, Chen, Hui, Liu, Weichen
Other Authors: School of Computer Science and Engineering
Format: Conference or Workshop Item
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/165387
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-165387
record_format dspace
spelling sg-ntu-dr.10356-1653872023-12-15T01:02:51Z You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms Luo, Xiangzhong Liu, Di Kong, Hao Huai, Shuo Chen, Hui Liu, Weichen School of Computer Science and Engineering 59th ACM/IEEE Design Automation Conference (DAC 2022) Parallel and Distributed Computing Centre HP-NTU Digital Manufacturing Corporate Lab Engineering::Computer science and engineering Neural Architecture Search Deep Neural Networks Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., \underline{\textit{you only search once}}). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods. Ministry of Education (MOE) Nanyang Technological University Submitted/Accepted version This work is partially supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2 (MOE2019-T2-1-071) and Tier 1 (MOE2019-T1-001-072), and partially supported by Nanyang Technological University, Singapore, under its NAP (M4082282) and SUG (M4082087). 2023-03-28T03:04:15Z 2023-03-28T03:04:15Z 2022 Conference Paper Luo, X., Liu, D., Kong, H., Huai, S., Chen, H. & Liu, W. (2022). You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms. 59th ACM/IEEE Design Automation Conference (DAC 2022), 475-480. https://dx.doi.org/10.1145/3489517.3530488 978-1-4503-9142-9 https://hdl.handle.net/10356/165387 10.1145/3489517.3530488 475 480 en MOE2019-T2-1-071 MOE2019-T1- 001-072 NAP (M4082282) SUG (M4082087) 10.21979/N9/2J9M9I © 2022 Association for Computing Machinery. All rights reserved. This paper was published in Proceedings of the 59th ACM/IEEE Design Automation Conference (DAC 2022) and is made available with permission of Association for Computing Machinery. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Neural Architecture Search
Deep Neural Networks
spellingShingle Engineering::Computer science and engineering
Neural Architecture Search
Deep Neural Networks
Luo, Xiangzhong
Liu, Di
Kong, Hao
Huai, Shuo
Chen, Hui
Liu, Weichen
You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
description Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., \underline{\textit{you only search once}}). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Luo, Xiangzhong
Liu, Di
Kong, Hao
Huai, Shuo
Chen, Hui
Liu, Weichen
format Conference or Workshop Item
author Luo, Xiangzhong
Liu, Di
Kong, Hao
Huai, Shuo
Chen, Hui
Liu, Weichen
author_sort Luo, Xiangzhong
title You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
title_short You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
title_full You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
title_fullStr You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
title_full_unstemmed You only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
title_sort you only search once: on lightweight differentiable architecture search for resource-constrained embedded platforms
publishDate 2023
url https://hdl.handle.net/10356/165387
_version_ 1787136505068650496