Graph convolution network based skeleton action recognition with DCT features

Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Grap...

Full description

Saved in:
Bibliographic Details
Main Author: Hei, Hao
Other Authors: Alex Chichung Kot
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172751
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172751
record_format dspace
spelling sg-ntu-dr.10356-1727512023-12-22T15:42:38Z Graph convolution network based skeleton action recognition with DCT features Hei, Hao Alex Chichung Kot School of Electrical and Electronic Engineering Rapid-Rich Object Search (ROSE) Lab EACKOT@ntu.edu.sg Engineering::Electrical and electronic engineering Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Graph Convolutional Networks (GCNs) on skeleton modality, little attention has been paid to taking advantage of the frequency representation of skeleton data. In this project, our objective is to study the effect of utilizing skeleton features in the frequency domain to perform HAR with GCN. To achieve the target, we first conduct a thorough review of current approaches for HAR and frequency analysis. Inspired by research on attention mechanism, we proposed to combine channel attention and 2-D Discrete Cosine Transform (DCT) as a universal layer of a deep learning network to utilize the frequency information from skeleton data, which can be inserted in the current GCNs for improvements in classification accuracy. With the NTU-RGBD dataset, we conducted the experiments on three advanced GCN-based models as baseline models. Analysis of the experiment results has proven that by adding the proposed network layer, the classification accuracy of human actions of all three baseline models improved. The enhanced performance indicates the effectiveness of frequency information in the task of skeleton action recognition, as well as the potential of attention mechanism in utilizing the frequency information. Bachelor of Engineering (Electrical and Electronic Engineering) 2023-12-19T07:59:15Z 2023-12-19T07:59:15Z 2023 Final Year Project (FYP) Hei, H. (2023). Graph convolution network based skeleton action recognition with DCT features. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/172751 https://hdl.handle.net/10356/172751 en A3297-222 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
spellingShingle Engineering::Electrical and electronic engineering
Hei, Hao
Graph convolution network based skeleton action recognition with DCT features
description Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Graph Convolutional Networks (GCNs) on skeleton modality, little attention has been paid to taking advantage of the frequency representation of skeleton data. In this project, our objective is to study the effect of utilizing skeleton features in the frequency domain to perform HAR with GCN. To achieve the target, we first conduct a thorough review of current approaches for HAR and frequency analysis. Inspired by research on attention mechanism, we proposed to combine channel attention and 2-D Discrete Cosine Transform (DCT) as a universal layer of a deep learning network to utilize the frequency information from skeleton data, which can be inserted in the current GCNs for improvements in classification accuracy. With the NTU-RGBD dataset, we conducted the experiments on three advanced GCN-based models as baseline models. Analysis of the experiment results has proven that by adding the proposed network layer, the classification accuracy of human actions of all three baseline models improved. The enhanced performance indicates the effectiveness of frequency information in the task of skeleton action recognition, as well as the potential of attention mechanism in utilizing the frequency information.
author2 Alex Chichung Kot
author_facet Alex Chichung Kot
Hei, Hao
format Final Year Project
author Hei, Hao
author_sort Hei, Hao
title Graph convolution network based skeleton action recognition with DCT features
title_short Graph convolution network based skeleton action recognition with DCT features
title_full Graph convolution network based skeleton action recognition with DCT features
title_fullStr Graph convolution network based skeleton action recognition with DCT features
title_full_unstemmed Graph convolution network based skeleton action recognition with DCT features
title_sort graph convolution network based skeleton action recognition with dct features
publisher Nanyang Technological University
publishDate 2023
url https://hdl.handle.net/10356/172751
_version_ 1787136534171877376