Combining adaptive hierarchical depth motion maps with skeletal joints for human action recognition

• Main Authors: Ding, Runwei, He, Qinqin, Liu, Hong, Liu, Mengyuan
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2019
• Subjects: Action Recognition; Depth Data; DRNTU::Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/100166; http://hdl.handle.net/10220/48566
• Institution: Nanyang Technological University

This paper presents a new framework for human action recognition by fusing human motion with skeletal joints. First, adaptive hierarchical depth motion maps (AH-DMMs) are proposed to capture the shape and motion cues of action sequences. Specifically, AH-DMMs are calculated over adaptive hierarchical windows and Gabor filters are used to encode the texture information of AH-DMMs. Then, spatial distances of skeletal joint positions are computed to characterize the structure information of the human body. Finally, two types of fusion methods including feature-level fusion and decision-level fusion are employed to combine the motion cues and structure information. The experimental results on public benchmark datasets, i.e., MSRAction3D and UTKinect-Action, show the effectiveness of the proposed method.