Stitching weight-shared deep neural networks for efficient multitask inference on GPU

• Main Authors: WANG, Zeyu, HE, Xiaoxi, ZHOU, Zimu, WANG, Xu, MA, Qiang, MIAO, Xin, LIU, Zhuo, THIELE, Lothar, YANG, Zheng.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Deep Neural Networks; Multitask Inference; Model Acceleration; OS and Networks; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7486; https://ink.library.smu.edu.sg/context/sis_research/article/8489/viewcontent/secon22_wang.pdf
• Institution: Singapore Management University

Intelligent personal and home applications demand multiple deep neural networks (DNNs) running on resourceconstrained platforms for compound inference tasks, known as multitask inference. To fit multiple DNNs into low-resource devices, emerging techniques resort to weight sharing among DNNs to reduc...