A comparison of Filipino and Japanese facial expressions and hand gestures in relation to affective states in programming sessions

• Main Authors: Tiam-Lee, Thomas James Z., Sumi, Kaoru
• Format: text
• Published: Animo Repository 2018
• Subjects: Face perception; Gesture recognition (Computer science); Emotion recognition; Computer Sciences
• Online Access: https://animorepository.dlsu.edu.ph/faculty_research/13275
• Institution: De La Salle University

This paper presents an analysis of Filipino and Japanese students’ facial expressions and hand gestures while solving a series of computer programming exercises. Frequently occurring Facial Action Coding System (FACS) muscle movements such as the widening of the eye (AU-5) and sucking of the lip (AU-28), as well as the occurrence of hand gestures, are identified and discussed in relation to the reported affective states of engagement and confusion. We show that it is possible to build models for predicting engagement and confusion using facial expressions and typing information. We believe that this study can help in the development of affectively aware computer systems for learning programming that can work with different cultures.