Large-scale analysis of non-termination bugs in real-world OSS projects
Termination is a crucial program property. Non-termination bugs can be subtle to detect and may remain hidden for long before they take effect. Many real-world programs still suffer from vast consequences (e.g., no response) caused by non-termination bugs. As a classic problem, termination proving h...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2022
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/7496 https://ink.library.smu.edu.sg/context/sis_research/article/8499/viewcontent/Shi2022LSA.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-8499 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-84992022-11-21T05:45:07Z Large-scale analysis of non-termination bugs in real-world OSS projects SHI, Xiuhan XIE, Xiaofei LI, Yi ZHANG, Yao CHEN, Sen LI, Xiaohong Termination is a crucial program property. Non-termination bugs can be subtle to detect and may remain hidden for long before they take effect. Many real-world programs still suffer from vast consequences (e.g., no response) caused by non-termination bugs. As a classic problem, termination proving has been studied for many years. Many termination checking tools and techniques have been developed and demonstrated effectiveness on existing wellestablished benchmarks. However, the capability of these tools in finding practical non-termination bugs has yet to be tested on real-world projects. To fill in this gap, in this paper, we conducted the first large-scale empirical study of non-termination bugs in real-world OSS projects. Specifically, we first devoted substantial manual efforts in collecting and analyzing 445 non-termination bugs from 3,142 GitHub commits and provided a systematic classification of the bugs based on their root causes. We constructed a new benchmark set characterizing the real-world bugs with simplified programs, including a non-termination dataset with 56 real and reproducible non-termination bugs and a termination dataset with 58 fixed programs. With the constructed benchmark, we evaluated five state-of-the-art termination analysis tools. The results show that the capabilities of the tested tools to make correct verdicts have obviously dropped compared with the existing benchmarks. Meanwhile, we identified the challenges and limitations that these tools face by analyzing the root causes of their unhandled bugs. Finally, we summarized the challenges and future research directions for detecting non-termination bugs in real-world projects. 2022-11-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7496 info:doi/10.1145/3540250.3549129 https://ink.library.smu.edu.sg/context/sis_research/article/8499/viewcontent/Shi2022LSA.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Non-termination Bug Benchmarking Empirical Study Programming Languages and Compilers Software Engineering |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Non-termination Bug Benchmarking Empirical Study Programming Languages and Compilers Software Engineering |
spellingShingle |
Non-termination Bug Benchmarking Empirical Study Programming Languages and Compilers Software Engineering SHI, Xiuhan XIE, Xiaofei LI, Yi ZHANG, Yao CHEN, Sen LI, Xiaohong Large-scale analysis of non-termination bugs in real-world OSS projects |
description |
Termination is a crucial program property. Non-termination bugs can be subtle to detect and may remain hidden for long before they take effect. Many real-world programs still suffer from vast consequences (e.g., no response) caused by non-termination bugs. As a classic problem, termination proving has been studied for many years. Many termination checking tools and techniques have been developed and demonstrated effectiveness on existing wellestablished benchmarks. However, the capability of these tools in finding practical non-termination bugs has yet to be tested on real-world projects. To fill in this gap, in this paper, we conducted the first large-scale empirical study of non-termination bugs in real-world OSS projects. Specifically, we first devoted substantial manual efforts in collecting and analyzing 445 non-termination bugs from 3,142 GitHub commits and provided a systematic classification of the bugs based on their root causes. We constructed a new benchmark set characterizing the real-world bugs with simplified programs, including a non-termination dataset with 56 real and reproducible non-termination bugs and a termination dataset with 58 fixed programs. With the constructed benchmark, we evaluated five state-of-the-art termination analysis tools. The results show that the capabilities of the tested tools to make correct verdicts have obviously dropped compared with the existing benchmarks. Meanwhile, we identified the challenges and limitations that these tools face by analyzing the root causes of their unhandled bugs. Finally, we summarized the challenges and future research directions for detecting non-termination bugs in real-world projects. |
format |
text |
author |
SHI, Xiuhan XIE, Xiaofei LI, Yi ZHANG, Yao CHEN, Sen LI, Xiaohong |
author_facet |
SHI, Xiuhan XIE, Xiaofei LI, Yi ZHANG, Yao CHEN, Sen LI, Xiaohong |
author_sort |
SHI, Xiuhan |
title |
Large-scale analysis of non-termination bugs in real-world OSS projects |
title_short |
Large-scale analysis of non-termination bugs in real-world OSS projects |
title_full |
Large-scale analysis of non-termination bugs in real-world OSS projects |
title_fullStr |
Large-scale analysis of non-termination bugs in real-world OSS projects |
title_full_unstemmed |
Large-scale analysis of non-termination bugs in real-world OSS projects |
title_sort |
large-scale analysis of non-termination bugs in real-world oss projects |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2022 |
url |
https://ink.library.smu.edu.sg/sis_research/7496 https://ink.library.smu.edu.sg/context/sis_research/article/8499/viewcontent/Shi2022LSA.pdf |
_version_ |
1770576357819940864 |