Deformable pose traversal convolution for 3D action and gesture recognition

The representation of 3D pose plays a critical role for 3D action and gesture recognition. Rather than representing a 3D pose directly by its joint locations, in this paper, we propose a Deformable Pose Traversal Convolution Network that applies one-dimensional convolution to traverse the 3D pose fo...

Full description

Saved in:
Bibliographic Details
Main Authors: Weng, Junwu, Liu, Mengyuan, Jiang, Xudong, Yuan, Junsong
Other Authors: School of Electrical and Electronic Engineering
Format: Conference or Workshop Item
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/140842
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-140842
record_format dspace
spelling sg-ntu-dr.10356-1408422020-09-26T21:53:34Z Deformable pose traversal convolution for 3D action and gesture recognition Weng, Junwu Liu, Mengyuan Jiang, Xudong Yuan, Junsong School of Electrical and Electronic Engineering 15th European Conference on Computer Vision 2018 Institute for Media Innovation (IMI) Engineering::Computer science and engineering Pose Traversal Pose Convolution The representation of 3D pose plays a critical role for 3D action and gesture recognition. Rather than representing a 3D pose directly by its joint locations, in this paper, we propose a Deformable Pose Traversal Convolution Network that applies one-dimensional convolution to traverse the 3D pose for its representation. Instead of fixing the receptive field when performing traversal convolution, it optimizes the convolution kernel for each joint, by considering contextual joints with various weights. This deformable convolution better utilizes the contextual joints for action and gesture recognition and is more robust to noisy joints. Moreover, by feeding the learned pose feature to a LSTM, we perform end-to-end training that jointly optimizes 3D pose representation and temporal sequence recognition. Experiments on three benchmark datasets validate the competitive performance of our proposed method, as well as its efficiency and robustness to handle noisy joints of pose. NRF (Natl Research Foundation, S’pore) Accepted version 2020-06-02T07:15:01Z 2020-06-02T07:15:01Z 2018 Conference Paper Weng, J., Liu, M., Jiang, X., & Yuan, J. (2018). Deformable pose traversal convolution for 3D action and gesture recognition. Proceedings of Computer Vision – 15th European Conference on Computer Vision 2018, 142-157. doi:10.1007/978-3-030-01234-2_9 https://hdl.handle.net/10356/140842 10.1007/978-3-030-01234-2_9 142 157 en © 2018 Springer Nature Switzerland AG. This is a post-peer-review, pre-copyedit version of an article published in Proceedings of Computer Vision – 15th European Conference on Computer Vision 2018. The final authenticated version is available online at: https://doi.org/10.1007/978-3-030-01234-2_9 application/pdf
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Pose Traversal
Pose Convolution
spellingShingle Engineering::Computer science and engineering
Pose Traversal
Pose Convolution
Weng, Junwu
Liu, Mengyuan
Jiang, Xudong
Yuan, Junsong
Deformable pose traversal convolution for 3D action and gesture recognition
description The representation of 3D pose plays a critical role for 3D action and gesture recognition. Rather than representing a 3D pose directly by its joint locations, in this paper, we propose a Deformable Pose Traversal Convolution Network that applies one-dimensional convolution to traverse the 3D pose for its representation. Instead of fixing the receptive field when performing traversal convolution, it optimizes the convolution kernel for each joint, by considering contextual joints with various weights. This deformable convolution better utilizes the contextual joints for action and gesture recognition and is more robust to noisy joints. Moreover, by feeding the learned pose feature to a LSTM, we perform end-to-end training that jointly optimizes 3D pose representation and temporal sequence recognition. Experiments on three benchmark datasets validate the competitive performance of our proposed method, as well as its efficiency and robustness to handle noisy joints of pose.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Weng, Junwu
Liu, Mengyuan
Jiang, Xudong
Yuan, Junsong
format Conference or Workshop Item
author Weng, Junwu
Liu, Mengyuan
Jiang, Xudong
Yuan, Junsong
author_sort Weng, Junwu
title Deformable pose traversal convolution for 3D action and gesture recognition
title_short Deformable pose traversal convolution for 3D action and gesture recognition
title_full Deformable pose traversal convolution for 3D action and gesture recognition
title_fullStr Deformable pose traversal convolution for 3D action and gesture recognition
title_full_unstemmed Deformable pose traversal convolution for 3D action and gesture recognition
title_sort deformable pose traversal convolution for 3d action and gesture recognition
publishDate 2020
url https://hdl.handle.net/10356/140842
_version_ 1681059672313823232