Legion: Massively composing rankers for improved bug localization at Adobe

• Main Authors: JARMAN, Darryl, BERRY, Jeffrey, SMITH, Riley, THUNG, Ferdian, LO, David
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Bug localization; information retrieval; bug reports; data augmentation; ranker composition; industrial study; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7636; https://ink.library.smu.edu.sg/context/sis_research/article/8639/viewcontent/09415126.pdf
• Institution: Singapore Management University

Studies have estimated that, in industrial settings, developers spend between 30 and 90 percent of their time fixing bugs. As such, tools that assist in identifying the location of bugs provide value by reducing debugging costs. One such tool is BugLocator. This study initially aimed to determine if developers working on the Adobe Analytics product could use BugLocator. The initial results show that BugLocator achieves a similar accuracy on five of seven Adobe Analytics repositories and on open-source projects. However, these results do not meet the minimum applicability requirement deemed necessary by Adobe Analytics developers prior to possible adoption. Thus, we consequently examine how BugLocator can achieve the targeted accuracy with two extensions: (1) adding more data corpora, and (2) massively composing individual rankers consisting of augmented BugLocator instances trained on various combinations of corpora and parameter configurations with a Random Forest model. We refer to our final extension as Legion. On average, applying Legion to Adobe Analytics repositories results in at least one buggy file ranked in the top-ten recommendations 76.8 percent of the time for customer-reported bugs across all 7 repositories. This represents a substantial improvement over BugLocator of 36.4 percent, and satisfies the minimum applicability requirement. Additionally, our extensions boost Mean Average Precision by 107.7 percent, Mean Reciprocal Rank by 86.1 percent, Top 1 by 143.4 percent and Top 5 by 58.1 percent.