AP-coach: Formative feedback generation for learning introductory programming concepts

• Main Authors: TA, Nguyen Binh Duong, SHAR, Lwin Khin, SHANKARARAMAN, Venky
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: learning programming; formative feedback; machine translation; code similarity.; Programming Languages and Compilers; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7622; https://doi.org/10.1109/TALE54877.2022.00060
• Institution: Singapore Management University

In this work, we aim to improve code writing skill in Python-based introductory programming courses for first-year university students. In such courses, students as novice programmers would benefit from personalised and formative feedback to: 1) quickly identify issues in their computational thinking process or coding techniques, and 2) know how to proceed when facing a certain problem. Due to the large number of students, it is impractical for instructors to manually assess all the work of each student to provide tailored feedback. We design and implement Automatic Programming Coach (AP-Coach), a web-based tool for automatically generating formative feedback for exercises on basic programming concepts. AP-Coach combines software engineering techniques (code similarity measures based on abstract syntax trees, and unit testing), and AI techniques (machine translation) in a novel manner to provide relevant feedback. We report promising results for AP-Coach in the following aspects: 1) quantitative evaluation of code similarity computation and machine translation, 2) qualitative evaluation of the perceived quality and usability of auto-generated feedback, and 3) experience of a selected group of computing students using the system.