Duplicate bug report detection: How far are we?
Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned t...
Saved in:
Main Authors: | , , , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2023
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/7788 https://ink.library.smu.edu.sg/context/sis_research/article/8791/viewcontent/3576042_pvoa.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-8791 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-87912023-09-13T00:48:09Z Duplicate bug report detection: How far are we? ZHANG, Ting HAN, DongGyun VINAYAKARAO, Venkatesh IRSAN, Ivana Clairine XU, Bowen Ferdian, Thung LO, David JIANG, Lingxiao Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in Mozilla and VSCode. Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research. 2023-07-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7788 info:doi/10.1145/3576042 https://ink.library.smu.edu.sg/context/sis_research/article/8791/viewcontent/3576042_pvoa.pdf http://creativecommons.org/licenses/by/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Bug Reports Duplicate Bug Report Detection Deep Learning Empirical Study Software Engineering |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Bug Reports Duplicate Bug Report Detection Deep Learning Empirical Study Software Engineering |
spellingShingle |
Bug Reports Duplicate Bug Report Detection Deep Learning Empirical Study Software Engineering ZHANG, Ting HAN, DongGyun VINAYAKARAO, Venkatesh IRSAN, Ivana Clairine XU, Bowen Ferdian, Thung LO, David JIANG, Lingxiao Duplicate bug report detection: How far are we? |
description |
Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in Mozilla and VSCode. Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research. |
format |
text |
author |
ZHANG, Ting HAN, DongGyun VINAYAKARAO, Venkatesh IRSAN, Ivana Clairine XU, Bowen Ferdian, Thung LO, David JIANG, Lingxiao |
author_facet |
ZHANG, Ting HAN, DongGyun VINAYAKARAO, Venkatesh IRSAN, Ivana Clairine XU, Bowen Ferdian, Thung LO, David JIANG, Lingxiao |
author_sort |
ZHANG, Ting |
title |
Duplicate bug report detection: How far are we? |
title_short |
Duplicate bug report detection: How far are we? |
title_full |
Duplicate bug report detection: How far are we? |
title_fullStr |
Duplicate bug report detection: How far are we? |
title_full_unstemmed |
Duplicate bug report detection: How far are we? |
title_sort |
duplicate bug report detection: how far are we? |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2023 |
url |
https://ink.library.smu.edu.sg/sis_research/7788 https://ink.library.smu.edu.sg/context/sis_research/article/8791/viewcontent/3576042_pvoa.pdf |
_version_ |
1779157131033837568 |