DeepArc: Modularizing neural networks for the model maintenance
Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure a...
Saved in:
Main Authors: | , , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2023
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/9316 https://ink.library.smu.edu.sg/context/sis_research/article/10316/viewcontent/icse23_DeepArc_av.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-10316 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-103162024-09-26T07:56:42Z DeepArc: Modularizing neural networks for the model maintenance REN, Xiaoning LIN, Yun XUE, Yinxing LIU, Ruofan SUN, Jun FENG, Zhiyong DONG, Jinsong Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure and re-adaption. In this work, we propose DeepArc, a novel modularization method for neural networks, to reduce the cost of model maintenance tasks. Specifically, DeepArc decomposes a neural network into several consecutive modules, each of which encapsulates consecutive layers with similar semantics. The network modularization facilitates practical tasks such as refactoring the model to preserve existing features (e.g., model compression) and enhancing the model with new features (e.g., fitting new samples). The modularization and encapsulation allow us to restructure or retrain the model by only pruning and tuning a few localized neurons and layers. Our experiments show that (1) DeepArc can boost the runtime efficiency of the state-of-the-art model compression techniques by 14.8%; (2) compared to the traditional model retraining, DeepArc only needs to train less than 20% of the neurons on average to fit adversarial samples and repair under-performing models, leading to 32.85% faster training performance while achieving similar model prediction performance. 2023-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/9316 info:doi/10.1109/ICSE48619.2023.00092 https://ink.library.smu.edu.sg/context/sis_research/article/10316/viewcontent/icse23_DeepArc_av.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University architecture modularization neural networks Software Engineering |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
architecture modularization neural networks Software Engineering |
spellingShingle |
architecture modularization neural networks Software Engineering REN, Xiaoning LIN, Yun XUE, Yinxing LIU, Ruofan SUN, Jun FENG, Zhiyong DONG, Jinsong DeepArc: Modularizing neural networks for the model maintenance |
description |
Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure and re-adaption. In this work, we propose DeepArc, a novel modularization method for neural networks, to reduce the cost of model maintenance tasks. Specifically, DeepArc decomposes a neural network into several consecutive modules, each of which encapsulates consecutive layers with similar semantics. The network modularization facilitates practical tasks such as refactoring the model to preserve existing features (e.g., model compression) and enhancing the model with new features (e.g., fitting new samples). The modularization and encapsulation allow us to restructure or retrain the model by only pruning and tuning a few localized neurons and layers. Our experiments show that (1) DeepArc can boost the runtime efficiency of the state-of-the-art model compression techniques by 14.8%; (2) compared to the traditional model retraining, DeepArc only needs to train less than 20% of the neurons on average to fit adversarial samples and repair under-performing models, leading to 32.85% faster training performance while achieving similar model prediction performance. |
format |
text |
author |
REN, Xiaoning LIN, Yun XUE, Yinxing LIU, Ruofan SUN, Jun FENG, Zhiyong DONG, Jinsong |
author_facet |
REN, Xiaoning LIN, Yun XUE, Yinxing LIU, Ruofan SUN, Jun FENG, Zhiyong DONG, Jinsong |
author_sort |
REN, Xiaoning |
title |
DeepArc: Modularizing neural networks for the model maintenance |
title_short |
DeepArc: Modularizing neural networks for the model maintenance |
title_full |
DeepArc: Modularizing neural networks for the model maintenance |
title_fullStr |
DeepArc: Modularizing neural networks for the model maintenance |
title_full_unstemmed |
DeepArc: Modularizing neural networks for the model maintenance |
title_sort |
deeparc: modularizing neural networks for the model maintenance |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2023 |
url |
https://ink.library.smu.edu.sg/sis_research/9316 https://ink.library.smu.edu.sg/context/sis_research/article/10316/viewcontent/icse23_DeepArc_av.pdf |
_version_ |
1814047907574185984 |