Skeleton-based human action recognition with graph neural networks

• Main Author: U S Vaitesswar
• Other Authors: Yeo Chai Kiat
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/156866
• Institution: Nanyang Technological University

Skeleton-based action recognition is a long-standing task in computer vision which aims to distinguish different human actions by identifying their unique characteristic patterns in the input data. Most of the existing GCN-based models developed for this task primarily model the skeleton graph as either directed or undirected. Furthermore, these models also restrict the receptive field in the temporal domain to a fixed range which significantly inhibits their expressibility. Therefore, a mixed graph network comprising both directed and undirected graph networks with a multi-range temporal module called MMGCN is proposed. In this way, the model can benefit from the different interpretations of the same action by the different graphs. Adding on, the multi-range temporal module enhances the model’s expressibility as it can choose the appropriate receptive field for each layer, thus allowing the model to dynamically adapt to the input data. With this lightweight MMGCN model, it is shown that deep learning models can learn the underlying patterns in the data and model large receptive fields without additional semantics or high model complexity. Finally, this model achieved state-of-the-art results on benchmark datasets: NTU-RGB+D, NTU-RGB+D 120, Skeleton-Kinetics and Northwestern-UCLA despite its low model complexity thus proving its effectiveness. An additional study was conducted to weigh the importance of model complexity (i.e. more nuanced architecture) against ensemble model learning (i.e. multiple input streams). The insights derived from this study will be useful for future models developed for skeleton-based action recognition task.