Recognition of student engagement and affective states using ConvNeXtlarge and ensemble GRU in E-Learning

• Main Authors: Shiri, Farhad Mortezapour, Perumal, Thinagaran, Mustapha, Norwati, Mohamed, Raihani, Ahmadon, Mohd Anuaruddin, Yamaguchi, Shingo
• Format: Book Section
• Language: English
• Published: Institute of Electrical and Electronics Engineers 2024
• Online Access: http://psasir.upm.edu.my/id/eprint/115331/1/115331.pdf; http://psasir.upm.edu.my/id/eprint/115331/; https://ieeexplore.ieee.org/document/10542707/
• Institution: Universiti Putra Malaysia

The advent of online learning has revolutionized the educational landscape, significantly enhancing global access to and affordability of education. However, a critical challenge in this domain is the efficacy of online classes, particularly in the context of the feedback loop between instructors and students. A pivotal aspect of this challenge is the assessment of student engagement and other affective states in online learning environments, a task that is inherently complex due to its nuanced nature. In this paper, we introduce a novel spatio-temporal hybrid deep learning model that combines the strengths of ConvNeXtLarge with an ensemble of gated recurrent unit (GRU and Bi-GRU) models to detect and classify affective states of students from video data. This model adeptly handles the multi-label, multi-class nature of the problem, employing four parallel fully connected (Dense) layers for effective classification. Our model's performance is evaluated using the DAiSEE dataset, a publicly available resource that captures a range of student affective states in an online learning environment. The proposed model demonstrates promising results in identifying four key affective states of boredom, engagement, confusion, and frustration, with accuracy rates of 54.89%, 56.46%, 70.18%, and 79.32% respectively. These findings underscore the efficacy of our model in accurately gauging student affective states, thereby offering significant insights into the potential of spatio-temporal hybrid networks in this realm. The study demonstrates the feasibility of employing such models to enhance the quality of online education.