Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks

With the incidence of the Covid-19 pandemic, institutions have adopted online learning as the main lessondelivery channel. A common criticism of online learning is that sensing of learners’ affective states such asengagement is lacking which degrades the quality of teaching. In this study, we propos...

Full description

Saved in:
Bibliographic Details
Main Author: FWA, Hua Leong
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2022
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7157
https://ink.library.smu.edu.sg/context/sis_research/article/8160/viewcontent/paper10_emotion_detection_STGAN_camera.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8160
record_format dspace
spelling sg-smu-ink.sis_research-81602023-08-04T06:01:02Z Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks FWA, Hua Leong With the incidence of the Covid-19 pandemic, institutions have adopted online learning as the main lessondelivery channel. A common criticism of online learning is that sensing of learners’ affective states such asengagement is lacking which degrades the quality of teaching. In this study, we propose automatic sensing of learners’ affective states in an online setting with web cameras capturing their facial landmarks and head poses. We postulate that the sparsely connected facial landmarks can be modelled using a Graph Neural Network. Using the publicly available in the wild DAiSEE dataset, we modelled both the spatial and temporal dimensions of the facial videos with a deep learning architecture consisting of Graph Attention Networks and Gated Recurrent Units. The ablation study confirmed that the differencing of consecutive frames of facial landmarks and the addition of head poses enhance the detection performance. The results further demonstrated that the model performed well in comparison with other models and more importantly, is suited for implementation on mobile devices with its low computational requirements. 2022-04-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7157 info:doi/10.5220/0010921200003182 https://ink.library.smu.edu.sg/context/sis_research/article/8160/viewcontent/paper10_emotion_detection_STGAN_camera.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University spatial temporal affective states facial landmarks graph attention network gated recurrent unit Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic spatial
temporal
affective states
facial landmarks
graph attention network
gated recurrent unit
Databases and Information Systems
spellingShingle spatial
temporal
affective states
facial landmarks
graph attention network
gated recurrent unit
Databases and Information Systems
FWA, Hua Leong
Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
description With the incidence of the Covid-19 pandemic, institutions have adopted online learning as the main lessondelivery channel. A common criticism of online learning is that sensing of learners’ affective states such asengagement is lacking which degrades the quality of teaching. In this study, we propose automatic sensing of learners’ affective states in an online setting with web cameras capturing their facial landmarks and head poses. We postulate that the sparsely connected facial landmarks can be modelled using a Graph Neural Network. Using the publicly available in the wild DAiSEE dataset, we modelled both the spatial and temporal dimensions of the facial videos with a deep learning architecture consisting of Graph Attention Networks and Gated Recurrent Units. The ablation study confirmed that the differencing of consecutive frames of facial landmarks and the addition of head poses enhance the detection performance. The results further demonstrated that the model performed well in comparison with other models and more importantly, is suited for implementation on mobile devices with its low computational requirements.
format text
author FWA, Hua Leong
author_facet FWA, Hua Leong
author_sort FWA, Hua Leong
title Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
title_short Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
title_full Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
title_fullStr Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
title_full_unstemmed Fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
title_sort fine-grained detection of academic emotions with spatial temporal graph attention networks using facial landmarks
publisher Institutional Knowledge at Singapore Management University
publishDate 2022
url https://ink.library.smu.edu.sg/sis_research/7157
https://ink.library.smu.edu.sg/context/sis_research/article/8160/viewcontent/paper10_emotion_detection_STGAN_camera.pdf
_version_ 1773551431433846784