How are deep learning models similar? An empirical study on clone analysis of deep learning software

• Main Authors: WU, Xiongfei, QIN, Liangyu, YU, Bing, XIE, Xiaofei, MA, Lei, XUE, Yinxing, LIU, Yang, ZHAO, Jianjun
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: code clone detection; deep learning; model similarity; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7111; https://ink.library.smu.edu.sg/context/sis_research/article/8114/viewcontent/3387904.3389254.pdf
• Institution: Singapore Management University

Deep learning (DL) has been successfully applied to many cutting-edge applications, e.g., image processing, speech recognition, and natural language processing. As more and more DL software is made open-sourced, publicly available, and organized in model repositories and stores (Model Zoo, ModelDepot), there comes a need to understand the relationships of these DL models regarding their maintenance and evolution tasks. Although clone analysis has been extensively studied for traditional software, up to the present, clone analysis has not been investigated for DL software. Since DL software adopts the data-driven development paradigm, it is still not clear whether and to what extent the clone analysis techniques of traditional software could be adapted to DL software.In this paper, we initiate the first step on the clone analysis of DL software at three different levels, i.e., source code level, model structural level, and input/output (I/0)-semantic level, which would be a key in DL software management, maintenance and evolution. We intend to investigate the similarity between these DL models from clone analysis perspective. Several tools and metrics are selected to conduct clone analysis of DL software at three different levels. Our study on two popular datasets (i.e., MNIST and CIFAR-10) and eight DL models of five architectural families (i.e., LeNet, ResNet, DenseNet, AlexNet, and VGG) shows that: 1). the three levels of similarity analysis are generally adequate to find clones between DL models ranging from structural to semantic; 2). different measures for clone analysis used at each level yield similar results; 3) clone analysis of one single level may not render a complete picture of the similarity of DL models. Our findings open up several research opportunities worth further exploration towards better understanding and more effective clone analysis of DL software.