Does deep learning improve the performance of duplicate bug report detection? An empirical study

• Main Authors: JIANG, Yuan, SU, Xiaohong, TREUDE, Christoph, SHANG, Chao, WANG, Tiantian
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Duplicate bug report detection; Deep learning; Information retrieval; Similarity measure; Realistic evaluation; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/8785; https://ink.library.smu.edu.sg/context/sis_research/article/9788/viewcontent/yuanjiang23.pdf
• Institution: Singapore Management University

Do Deep Learning (DL) techniques actually help to improve the performance of duplicate bug report detection? Prior studies suggest that they do, if the duplicate bug report detection task is treated as a binary classification problem. However, in realistic scenarios, the task is often viewed as a ranking problem, which predicts potential duplicate bug reports by ranking based on similarities with existing historical bug reports. There is little empirical evidence to support that DL can be effectively applied to detect duplicate bug reports in the ranking scenario. Therefore, in this paper, we investigate whether well-known DL-based methods outperform classic information retrieval (IR) based methods on the duplicate bug report detection task. In addition, we argue that both IR- and DL-based methods suffer from incompletely evaluating the similarity between bug reports, resulting in the loss of important information. To address this problem, we propose a new method that combines IR and DL techniques to compute textual similarity more comprehensively. Our experimental results show that the DL-based method itself does not yield high performance compared to IR-based methods. However, our proposed combined method improves on the MAP metric of classic IR-based methods by a median of 7.09%–11.34% and a maximum of 17.228%–28.97%.