Revisiting the identification of the co-evolution of production and test code

Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and...

Full description

Saved in:
Bibliographic Details
Main Authors: SUN, Weifeng, YAN, Meng, LIU, Zhongxin, XIA, Xin, LEI, Yan, LO, David
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8280
https://ink.library.smu.edu.sg/context/sis_research/article/9283/viewcontent/Revisit_Id_Co_Evolution_Prod_Test_Code_av.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9283
record_format dspace
spelling sg-smu-ink.sis_research-92832023-11-10T08:30:12Z Revisiting the identification of the co-evolution of production and test code SUN, Weifeng YAN, Meng LIU, Zhongxin XIA, Xin LEI, Yan LO, David Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and a production code change where the test code change triggers or is triggered by the production code change. The quality of the mined samples is critical to the reliability of research conclusions. Existing studies mined production-test co-evolution samples based on the following assumption: if a test class and its associated production class change together in one commit, or a test class changes immediately after the changes of the associated production class within a short time interval, this change pair should be a production-test co-evolution sample. However, the validity of this assumption has never been investigated.To fill this gap, we present an empirical study, investigating the reasons for test code updates occurring after the associated production code changes, and revealing the pervasive existence of noise in the production-test co-evolution samples identified based on the aforementioned assumption by existing works. We define a taxonomy of such noise, including six categories (i.e., adaptive maintenance, perfective maintenance, corrective maintenance, indirectly related production code update, indirectly related test code update, and other reasons). Guided by the empirical findings, we propose CHOSEN (an identifiCation metHod Of production-teSt co-EvolutioN) based on a two-stage strategy. CHOSEN takes a test code change and its associated production code change as input, aiming to determine whether the production-test change pair is a production-test co-evolution sample. Such identified samples are the basis of or are useful for various downstream tasks. We conduct a series of experiments to evaluate our method. Results show that (1) CHOSEN achieves an AUC of 0.931 and an F1-score of 0.928, significantly outperforming existing identification methods, and (2) CHOSEN can help researchers and practitioners draw more accurate conclusions on studies related to the co-evolution of production and test code. For the task of Just-In-Time (JIT) obsolete test code detection, which can help detect whether a piece of test code should be updated when developers modify the production code, the test set constructed by CHOSEN can help measure the detection method's performance more accurately, only leading to 0.76% of average error compared with ground truth. In addition, the dataset constructed by CHOSEN can be used to train a better obsolete test code detection model, of which the average improvements on accuracy, precision, recall, and F1-score are 12.00%, 17.35%, 8.75%, and 13.50% respectively. 2023-09-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8280 info:doi/10.1145/3607183 https://ink.library.smu.edu.sg/context/sis_research/article/9283/viewcontent/Revisit_Id_Co_Evolution_Prod_Test_Code_av.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Empirical software engineering mining software repositories software evolution software testing Software Engineering
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Empirical software engineering
mining software repositories
software evolution
software testing
Software Engineering
spellingShingle Empirical software engineering
mining software repositories
software evolution
software testing
Software Engineering
SUN, Weifeng
YAN, Meng
LIU, Zhongxin
XIA, Xin
LEI, Yan
LO, David
Revisiting the identification of the co-evolution of production and test code
description Many software processes advocate that the test code should co-evolve with the production code. Prior work usually studies such co-evolution based on production-test co-evolution samples mined from software repositories. A production-test co-evolution sample refers to a pair of a test code change and a production code change where the test code change triggers or is triggered by the production code change. The quality of the mined samples is critical to the reliability of research conclusions. Existing studies mined production-test co-evolution samples based on the following assumption: if a test class and its associated production class change together in one commit, or a test class changes immediately after the changes of the associated production class within a short time interval, this change pair should be a production-test co-evolution sample. However, the validity of this assumption has never been investigated.To fill this gap, we present an empirical study, investigating the reasons for test code updates occurring after the associated production code changes, and revealing the pervasive existence of noise in the production-test co-evolution samples identified based on the aforementioned assumption by existing works. We define a taxonomy of such noise, including six categories (i.e., adaptive maintenance, perfective maintenance, corrective maintenance, indirectly related production code update, indirectly related test code update, and other reasons). Guided by the empirical findings, we propose CHOSEN (an identifiCation metHod Of production-teSt co-EvolutioN) based on a two-stage strategy. CHOSEN takes a test code change and its associated production code change as input, aiming to determine whether the production-test change pair is a production-test co-evolution sample. Such identified samples are the basis of or are useful for various downstream tasks. We conduct a series of experiments to evaluate our method. Results show that (1) CHOSEN achieves an AUC of 0.931 and an F1-score of 0.928, significantly outperforming existing identification methods, and (2) CHOSEN can help researchers and practitioners draw more accurate conclusions on studies related to the co-evolution of production and test code. For the task of Just-In-Time (JIT) obsolete test code detection, which can help detect whether a piece of test code should be updated when developers modify the production code, the test set constructed by CHOSEN can help measure the detection method's performance more accurately, only leading to 0.76% of average error compared with ground truth. In addition, the dataset constructed by CHOSEN can be used to train a better obsolete test code detection model, of which the average improvements on accuracy, precision, recall, and F1-score are 12.00%, 17.35%, 8.75%, and 13.50% respectively.
format text
author SUN, Weifeng
YAN, Meng
LIU, Zhongxin
XIA, Xin
LEI, Yan
LO, David
author_facet SUN, Weifeng
YAN, Meng
LIU, Zhongxin
XIA, Xin
LEI, Yan
LO, David
author_sort SUN, Weifeng
title Revisiting the identification of the co-evolution of production and test code
title_short Revisiting the identification of the co-evolution of production and test code
title_full Revisiting the identification of the co-evolution of production and test code
title_fullStr Revisiting the identification of the co-evolution of production and test code
title_full_unstemmed Revisiting the identification of the co-evolution of production and test code
title_sort revisiting the identification of the co-evolution of production and test code
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/8280
https://ink.library.smu.edu.sg/context/sis_research/article/9283/viewcontent/Revisit_Id_Co_Evolution_Prod_Test_Code_av.pdf
_version_ 1783955663988195328