FFL: fine grained fault localization for student programs via syntactic and semantic reasoning

• Main Authors: NGUYEN, Thanh Dat, LE, Cong Thanh, LUONG, Duc-Minh, DUONG, Van-Hai, LE, Xuan Bach, LO, David, HUYNH, Quyet-Thang
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Fault localization; Programming education; Graph neural network; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7641; https://ink.library.smu.edu.sg/context/sis_research/article/8644/viewcontent/795600a151.pdf
• Institution: Singapore Management University

Fault localization has been used to provide feedback for incorrect student programs since locations of faults can be a valuable hint for students about what caused their programs to crash. Unfortunately, existing fault localization techniques for student programs are limited because they usually consider either the program’s syntax or semantics alone. This motivates the new design of fault localization techniques that use both semantic and syntactical information of the program. In this paper, we introduce FFL (Fine grained Fault Localization), a novel technique using syntactic and semantic reasoning for localizing bugs in student programs. The novelty in FFL that allows it to capture both syntactic and semantic of a program is three-fold: (1) A fine-grained graph-based representation of a program that is adaptive for statement-level fault localization; (2) an effective and efficient model to leverage the designed representation for fault-localization task and (3) a node-level training objective that allows deep learning model to learn from fine-grained syntactic patterns. We compare FFL’s effectiveness with state-of-the-art fault localization techniques for student programs (NBL, Tarantula, Ochiai and DStar) on two real-world datasets: Prutor and Codeflaws. Experimental results show that FFL successfully localizes bug for 84.6% out of 2136 programs on Prutor and 83.1% out of 780 programs on Codeflaws concerning the top-10 suspicious statements. FFL also remarkably outperforms the best baselines by 197%, 104%, 70%, 22% on Codeflaws dataset and 10%, 17%, 15% and 8% on Prutor dataset, in term of top-1, top-3, top-5, top-10, respectively