3D articulated skeleton extraction using a single consumer-grade depth camera

Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single con...

Full description

Saved in:
Bibliographic Details
Main Authors: Lu, Xuequan, Deng, Zhigang, Luo, Jun, Chen, Wenzhi, Yeung, Sai-Kit, He, Ying
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2021
Subjects:
Online Access:https://hdl.handle.net/10356/151523
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-151523
record_format dspace
spelling sg-ntu-dr.10356-1515232021-06-18T03:25:02Z 3D articulated skeleton extraction using a single consumer-grade depth camera Lu, Xuequan Deng, Zhigang Luo, Jun Chen, Wenzhi Yeung, Sai-Kit He, Ying School of Computer Science and Engineering Engineering::Computer science and engineering Unsupervised Skeleton Extraction Single View Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains challenging, since such data are often corrupted with substantial noise and outliers. Relatively few approaches have been proposed to tackle this problem. In this paper, we present a novel unsupervised framework to address this issue. Specifically, we first build one-to-one point correspondences among the point cloud frames in a sequence with our non-rigid point cloud registration algorithm. We then generate a skeleton involving a reasonable number of joints and bones with our skeletal structure extraction algorithm. We lastly present an iterative Linear Blend Skinning based algorithm for accurate joints learning. At the end, our method can learn a quality articulated skeleton from a single 3D point sequence possibly corrupted with noise and outliers. Through qualitative and quantitative evaluations on both publicly available data and in-house Kinect-captured data, we show that our unsupervised approach soundly outperforms state of the art techniques in terms of both quality (i.e., visual) and accuracy (i.e., Euclidean distance error metric). Moreover, the poses of our extracted skeletons are even comparable to those by KinectSDK, a well-known supervised pose estimation technique; for example, our method and KinectSDK achieves similar distance errors of 0.0497 and 0.0521. Ministry of Education (MOE) Xuequan Lu is supported in part by Deakin University, Australia CY01-251301-F003-PJ03906-PG00447. Zhigang Deng is in part supported by NSF, USA IIS-1524782. Jun Luo is supported in part by AcRF Tier 2 Grant MOE2016-T2-2-022 (Singapore). Ying He is supported by MOE RG26/17. Sai-Kit Yeung is supported by an internal grant from HKUST (R9429). 2021-06-18T03:25:02Z 2021-06-18T03:25:02Z 2019 Journal Article Lu, X., Deng, Z., Luo, J., Chen, W., Yeung, S. & He, Y. (2019). 3D articulated skeleton extraction using a single consumer-grade depth camera. Computer Vision and Image Understanding, 188, 102792-. https://dx.doi.org/10.1016/j.cviu.2019.102792 1077-3142 https://hdl.handle.net/10356/151523 10.1016/j.cviu.2019.102792 2-s2.0-85071291985 188 102792 en MOE2016-T2-2-022 MOE RG26/17 Computer Vision and Image Understanding © 2019 Elsevier Inc. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Unsupervised Skeleton Extraction
Single View
spellingShingle Engineering::Computer science and engineering
Unsupervised Skeleton Extraction
Single View
Lu, Xuequan
Deng, Zhigang
Luo, Jun
Chen, Wenzhi
Yeung, Sai-Kit
He, Ying
3D articulated skeleton extraction using a single consumer-grade depth camera
description Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains challenging, since such data are often corrupted with substantial noise and outliers. Relatively few approaches have been proposed to tackle this problem. In this paper, we present a novel unsupervised framework to address this issue. Specifically, we first build one-to-one point correspondences among the point cloud frames in a sequence with our non-rigid point cloud registration algorithm. We then generate a skeleton involving a reasonable number of joints and bones with our skeletal structure extraction algorithm. We lastly present an iterative Linear Blend Skinning based algorithm for accurate joints learning. At the end, our method can learn a quality articulated skeleton from a single 3D point sequence possibly corrupted with noise and outliers. Through qualitative and quantitative evaluations on both publicly available data and in-house Kinect-captured data, we show that our unsupervised approach soundly outperforms state of the art techniques in terms of both quality (i.e., visual) and accuracy (i.e., Euclidean distance error metric). Moreover, the poses of our extracted skeletons are even comparable to those by KinectSDK, a well-known supervised pose estimation technique; for example, our method and KinectSDK achieves similar distance errors of 0.0497 and 0.0521.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Lu, Xuequan
Deng, Zhigang
Luo, Jun
Chen, Wenzhi
Yeung, Sai-Kit
He, Ying
format Article
author Lu, Xuequan
Deng, Zhigang
Luo, Jun
Chen, Wenzhi
Yeung, Sai-Kit
He, Ying
author_sort Lu, Xuequan
title 3D articulated skeleton extraction using a single consumer-grade depth camera
title_short 3D articulated skeleton extraction using a single consumer-grade depth camera
title_full 3D articulated skeleton extraction using a single consumer-grade depth camera
title_fullStr 3D articulated skeleton extraction using a single consumer-grade depth camera
title_full_unstemmed 3D articulated skeleton extraction using a single consumer-grade depth camera
title_sort 3d articulated skeleton extraction using a single consumer-grade depth camera
publishDate 2021
url https://hdl.handle.net/10356/151523
_version_ 1703971250349539328