Neural architecture search as sparse supernet

Neural architecture search as sparse supernet

This paper aims at enlarging the problem of Neural Architecture Search (NAS) from Single-Path and Multi-Path Search to automated Mixed-Path Search. In particular, we model the NAS problem as a sparse supernet using a new continuous architecture representation with a mixture of sparsity constraints....

Full description

Saved in:
Bibliographic Details
Main Authors: WU, Y., LIU, A., HUANG, Zhiwu, ZHANG, S., VAN, Gool L.
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2021
Subjects:
OS and Networks
Systems Architecture
Online Access:https://ink.library.smu.edu.sg/sis_research/6411
https://ink.library.smu.edu.sg/context/sis_research/article/7414/viewcontent/Neural_architecture_search_as_sparse_supernet.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:This paper aims at enlarging the problem of Neural Architecture Search (NAS) from Single-Path and Multi-Path Search to automated Mixed-Path Search. In particular, we model the NAS problem as a sparse supernet using a new continuous architecture representation with a mixture of sparsity constraints. The sparse supernet enables us to automatically achieve sparsely-mixed paths upon a compact set of nodes. To optimize the proposed sparse supernet, we exploit a hierarchical accelerated proximal gradient algorithm within a bi-level optimization framework. Extensive experiments on Convolutional Neural Network and Recurrent Neural Network search demonstrate that the proposed method is capable of searching for compact, general and powerful neural architectures.

Similar Items