Enhancing developer interactions with programming screencasts through accurate code extraction

• Main Authors: BAO, Lingfeng, PAN, Shengyi, XING, Zhenchang, XIA, Xin, LO, David, YANG, Xiaohu
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: Programming; videos; code extraction; computer vision; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/5632; https://ink.library.smu.edu.sg/context/sis_research/article/6635/viewcontent/Enhancing_Developer_Interactions_pv.pdf
• Institution: Singapore Management University

Programming screencasts have become a pervasive resource on the Internet, which is favoured by many developers for learning new programming skills. For developers, the source code in screencasts is valuable and important. However, the streaming nature of screencasts limits the choice that they have for interacting with the code. Many studies apply the Optical Character Recognition (OCR) technique to convert screen images into text, which can be easily searched and indexed. However, we observe that the noise in the screen images significantly affects the quality of OCRed code.In this paper, we develop a tool named psc2code, which has two components, denoising code extraction from screencasts and enhancing programming video interaction. Experiment results on 1142 programming screencasts from YouTube show psc2code can effectively identify frames containing valid code region with a F1-score of 0.88 and improve the quality of OCRed code by fixing 46% of the errors. We also conduct a user study to evaluate the applicability of psc2code in enhancing video interaction, which shows it helps participants learn the knowledge in tutorials more efficiently.