3D articulated skeleton extraction using a single consumer-grade depth camera
Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single con...
Saved in:
Main Authors: | , , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2021
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/151523 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-151523 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1515232021-06-18T03:25:02Z 3D articulated skeleton extraction using a single consumer-grade depth camera Lu, Xuequan Deng, Zhigang Luo, Jun Chen, Wenzhi Yeung, Sai-Kit He, Ying School of Computer Science and Engineering Engineering::Computer science and engineering Unsupervised Skeleton Extraction Single View Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains challenging, since such data are often corrupted with substantial noise and outliers. Relatively few approaches have been proposed to tackle this problem. In this paper, we present a novel unsupervised framework to address this issue. Specifically, we first build one-to-one point correspondences among the point cloud frames in a sequence with our non-rigid point cloud registration algorithm. We then generate a skeleton involving a reasonable number of joints and bones with our skeletal structure extraction algorithm. We lastly present an iterative Linear Blend Skinning based algorithm for accurate joints learning. At the end, our method can learn a quality articulated skeleton from a single 3D point sequence possibly corrupted with noise and outliers. Through qualitative and quantitative evaluations on both publicly available data and in-house Kinect-captured data, we show that our unsupervised approach soundly outperforms state of the art techniques in terms of both quality (i.e., visual) and accuracy (i.e., Euclidean distance error metric). Moreover, the poses of our extracted skeletons are even comparable to those by KinectSDK, a well-known supervised pose estimation technique; for example, our method and KinectSDK achieves similar distance errors of 0.0497 and 0.0521. Ministry of Education (MOE) Xuequan Lu is supported in part by Deakin University, Australia CY01-251301-F003-PJ03906-PG00447. Zhigang Deng is in part supported by NSF, USA IIS-1524782. Jun Luo is supported in part by AcRF Tier 2 Grant MOE2016-T2-2-022 (Singapore). Ying He is supported by MOE RG26/17. Sai-Kit Yeung is supported by an internal grant from HKUST (R9429). 2021-06-18T03:25:02Z 2021-06-18T03:25:02Z 2019 Journal Article Lu, X., Deng, Z., Luo, J., Chen, W., Yeung, S. & He, Y. (2019). 3D articulated skeleton extraction using a single consumer-grade depth camera. Computer Vision and Image Understanding, 188, 102792-. https://dx.doi.org/10.1016/j.cviu.2019.102792 1077-3142 https://hdl.handle.net/10356/151523 10.1016/j.cviu.2019.102792 2-s2.0-85071291985 188 102792 en MOE2016-T2-2-022 MOE RG26/17 Computer Vision and Image Understanding © 2019 Elsevier Inc. All rights reserved. |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering Unsupervised Skeleton Extraction Single View |
spellingShingle |
Engineering::Computer science and engineering Unsupervised Skeleton Extraction Single View Lu, Xuequan Deng, Zhigang Luo, Jun Chen, Wenzhi Yeung, Sai-Kit He, Ying 3D articulated skeleton extraction using a single consumer-grade depth camera |
description |
Articulated skeleton extraction or learning has been extensively studied for 2D (e.g., images and video) and 3D (e.g., volume sequences, motion capture, and mesh sequences) data. Nevertheless, robustly and accurately learning 3D articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains challenging, since such data are often corrupted with substantial noise and outliers. Relatively few approaches have been proposed to tackle this problem. In this paper, we present a novel unsupervised framework to address this issue. Specifically, we first build one-to-one point correspondences among the point cloud frames in a sequence with our non-rigid point cloud registration algorithm. We then generate a skeleton involving a reasonable number of joints and bones with our skeletal structure extraction algorithm. We lastly present an iterative Linear Blend Skinning based algorithm for accurate joints learning. At the end, our method can learn a quality articulated skeleton from a single 3D point sequence possibly corrupted with noise and outliers. Through qualitative and quantitative evaluations on both publicly available data and in-house Kinect-captured data, we show that our unsupervised approach soundly outperforms state of the art techniques in terms of both quality (i.e., visual) and accuracy (i.e., Euclidean distance error metric). Moreover, the poses of our extracted skeletons are even comparable to those by KinectSDK, a well-known supervised pose estimation technique; for example, our method and KinectSDK achieves similar distance errors of 0.0497 and 0.0521. |
author2 |
School of Computer Science and Engineering |
author_facet |
School of Computer Science and Engineering Lu, Xuequan Deng, Zhigang Luo, Jun Chen, Wenzhi Yeung, Sai-Kit He, Ying |
format |
Article |
author |
Lu, Xuequan Deng, Zhigang Luo, Jun Chen, Wenzhi Yeung, Sai-Kit He, Ying |
author_sort |
Lu, Xuequan |
title |
3D articulated skeleton extraction using a single consumer-grade depth camera |
title_short |
3D articulated skeleton extraction using a single consumer-grade depth camera |
title_full |
3D articulated skeleton extraction using a single consumer-grade depth camera |
title_fullStr |
3D articulated skeleton extraction using a single consumer-grade depth camera |
title_full_unstemmed |
3D articulated skeleton extraction using a single consumer-grade depth camera |
title_sort |
3d articulated skeleton extraction using a single consumer-grade depth camera |
publishDate |
2021 |
url |
https://hdl.handle.net/10356/151523 |
_version_ |
1703971250349539328 |