Augmenting teacher noticing in science experiments: using computer vision to extract student activity information for science teachers

• Main Author: Chng, Edwin
• Other Authors: School of Mechanical and Aerospace Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Computer vision; Science teaching
• Online Access: https://hdl.handle.net/10356/181107; https://www.ntu.edu.sg/mae/ai-education-singapore-2024/activities/keynote-invited-talk#Content_C021_Col00
• Institution: Nanyang Technological University

During science experiments, teachers are limited in their ability to gather meaningful information about student activities. For example, teachers’ cognitive limit prevents them from managing numerous inputs from multiple students (Sherin and Star, 2011), and teachers’ student interaction limit prevents them from being aware of the intricacies of each student’s learning trajectory (Clark et al., 2012). To cope with these limitations, teachers tend to place an undue focus on the procedural steps taken by each student during science experiments (Wang et al., 2010). However, as underscored by Tang et al. (2010), such pedagogical behaviors can distract teachers from a more critical evaluation of students’ scientific thinking. Therefore, knowing students’ actions during science experiments represents a vital piece of information that can help nudge teachers towards the proper conduct of scientific inquiry. With this in mind, I propose the use of computer vision to extract student activity information for science teachers, so as to expand their ability to gather meaningful student information during science experiments. By working with science educators within Singapore’s education system, I examine how the envisioned computer vision system might function in a real-world setting. In this talk, I present qualitative findings on the design considerations for a computer vision system that provides instructional support in science experiments and share an action recognition system that has been constructed to fulfil this purpose. Overall, this work seeks to establish a preliminary understanding of how computer vision could be used as a tool to augment teacher noticing in science experiments.