Detecting C++ compiler front-end bugs via grammar mutation and differential testing

• Main Authors: TU, Haoxin, JIANG, He, ZHOU, Zhide, TANG, Yixuan, REN, Zhilei, QIAO, Lei, JIANG, Lingxiao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Computer bugs; C++ languages; Program processors; Grammar; Testing; Software; Syntactics; Automated testing; compiler defect; compiler testing; front-end; reliability; software testing; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7170; https://ink.library.smu.edu.sg/context/sis_research/article/8173/viewcontent/DetectingC_compilerFront_endBugs_av.pdf
• Institution: Singapore Management University

C++ is a widely used programming language and the C++ front-end is a critical part of a C++ compiler. Although many techniques have been proposed to test compilers, few studies are devoted to detecting bugs in C++ compiler. In this study, we take the first step to detect bugs in C++ compiler front-ends. To do so, two main challenges need to be addressed, namely, the acquisition of test programs that are more likely to trigger bugs in compiler front-ends and the bug identification from complicated compiler outputs. In this article, we propose a novel framework named Ccoft to detect bugs in C++ compiler front-ends. To address the first challenge, Ccoft implements a practical program generator. The generator first transforms C++ grammars into a flexible structured format and then utilizes an equal-chance selection (ECS) strategy to conduct structure-aware grammar mutation to generate diverse C++ programs. Next, Ccoft employs a set of differential testing strategies to identify various kinds of bugs in C++ compiler front-ends by comparing complex outputs emitted by C++ compilers, thus tackling the second challenge. Empirical evaluation results over two mainstream compilers (i.e., GCC and Clang) show that Ccoft greatly improves two state-of-the-art approaches (i.e., Dharma and Grammarinator) by 135% and 111% in terms of the numbers of detected bugs, respectively. By running Ccoft for three months, we have successfully reported 136 bugs for two C++ compilers, of which 78 (57 confirmed, assigned, or fixed) for GCC and 58 (10 confirmed or fixed) for Clang.