Is using deep learning frameworks free?: Characterizing technical debt in deep learning frameworks

• Main Authors: LIU, Jiakun, HUANG, Qiao, XIA, Xin, SHIHAB, Emad, LO, David, LI, Shanping
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: Software evolution; Maintaining software; self-admitted technical debt; deep learning; categorization; empirical study; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/5645; https://ink.library.smu.edu.sg/context/sis_research/article/6648/viewcontent/liu_icse2020.pdf
• Institution: Singapore Management University

Developers of deep learning applications (shortened as application developers) commonly use deep learning frameworks in their projects. However, due to time pressure, market competition, and cost reduction, developers of deep learning frameworks (shortened as framework developers) often have to sacrifice software quality to satisfy a shorter completion time. This practice leads to technical debt in deep learning frameworks, which results in the increasing burden to both the application developers and the framework developers in future development.In this paper, we analyze the comments indicating technical debt (self-admitted technical debt) in 7 of the most popular open-source deep learning frameworks. Although framework developers are aware of such technical debt, typically the application developers are not. We find that: 1) there is a significant number of technical debt in all the studied deep learning frameworks. 2) there is design debt, defect debt, documentation debt, test debt, requirement debt, compatibility debt, and algorithm debt in deep learning frameworks. 3) the majority of the technical debt in deep learning framework is design debt (24.07% - 65.27%), followed by requirement debt (7.09% - 31.48%) and algorithm debt (5.62% - 20.67%). In some projects, compatibility debt accounts for more than 10%. These findings illustrate that technical debt is common in deep learning frameworks, and many types of technical debt also impact the deep learning applications. Based on our findings, we highlight future research directions and provide recommendations for practitioners.