Variational relational point completion network for robust 3D classification

Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic...

Full description

Saved in:
Bibliographic Details
Main Authors: Pan, Liang, Chen, Xinyi, Cai, Zhongang, Zhang, Junzhe, Zhao, Haiyu, Yi, Shuai, Liu, Ziwei
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172185
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172185
record_format dspace
spelling sg-ntu-dr.10356-1721852023-11-28T07:28:28Z Variational relational point completion network for robust 3D classification Pan, Liang Chen, Xinyi Cai, Zhongang Zhang, Junzhe Zhao, Haiyu Yi, Shuai Liu, Ziwei School of Computer Science and Engineering S-Lab Engineering::Computer science and engineering 3D Perception Point Cloud Completion Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40 dataset) containing over 200,000 high-quality scans, which render partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-the-art methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans. Moreover, we can achieve robust 3D classification for partial point clouds with the help of VRCNet, which can highly increase classification accuracy. Ministry of Education (MOE) Nanyang Technological University This work was supported in part by the Ministry of Education, Singapore, under its MOE AcRF Tier 2 (MOE-T2EP20221-0012), NTU NAP, and in part by the RIE2020 Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s). 2023-11-28T07:28:28Z 2023-11-28T07:28:28Z 2023 Journal Article Pan, L., Chen, X., Cai, Z., Zhang, J., Zhao, H., Yi, S. & Liu, Z. (2023). Variational relational point completion network for robust 3D classification. IEEE Transactions On Pattern Analysis and Machine Intelligence, 45(9), 11340-11351. https://dx.doi.org/10.1109/TPAMI.2023.3268305 0162-8828 https://hdl.handle.net/10356/172185 10.1109/TPAMI.2023.3268305 37083514 2-s2.0-85153801950 9 45 11340 11351 en MOE-T2EP20221-0012 IEEE Transactions on Pattern Analysis and Machine Intelligence © 2023 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
3D Perception
Point Cloud Completion
spellingShingle Engineering::Computer science and engineering
3D Perception
Point Cloud Completion
Pan, Liang
Chen, Xinyi
Cai, Zhongang
Zhang, Junzhe
Zhao, Haiyu
Yi, Shuai
Liu, Ziwei
Variational relational point completion network for robust 3D classification
description Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40 dataset) containing over 200,000 high-quality scans, which render partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-the-art methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans. Moreover, we can achieve robust 3D classification for partial point clouds with the help of VRCNet, which can highly increase classification accuracy.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Pan, Liang
Chen, Xinyi
Cai, Zhongang
Zhang, Junzhe
Zhao, Haiyu
Yi, Shuai
Liu, Ziwei
format Article
author Pan, Liang
Chen, Xinyi
Cai, Zhongang
Zhang, Junzhe
Zhao, Haiyu
Yi, Shuai
Liu, Ziwei
author_sort Pan, Liang
title Variational relational point completion network for robust 3D classification
title_short Variational relational point completion network for robust 3D classification
title_full Variational relational point completion network for robust 3D classification
title_fullStr Variational relational point completion network for robust 3D classification
title_full_unstemmed Variational relational point completion network for robust 3D classification
title_sort variational relational point completion network for robust 3d classification
publishDate 2023
url https://hdl.handle.net/10356/172185
_version_ 1783955628142624768