Multi point-voxel convolution (MPVConv) for deep learning on point clouds

The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models...

Full description

Saved in:
Bibliographic Details
Main Authors: Zhou, Wei, Zhang, Xiaodan, Hao, Xingxing, Wang, Dekui, He, Ying
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172090
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172090
record_format dspace
spelling sg-ntu-dr.10356-1720902023-11-22T01:59:47Z Multi point-voxel convolution (MPVConv) for deep learning on point clouds Zhou, Wei Zhang, Xiaodan Hao, Xingxing Wang, Dekui He, Ying School of Computer Science and Engineering Engineering::Computer science and engineering 3D Deep Learning Point Clouds The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN and DRINet, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to 36%, and achieves higher accuracy than the voxel-based model with up to 34× speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to 8× speedups. Also, our MPVConv only needs 65% of the GPU memory required by the latest point-voxel-based model (DRINet). The source code of our method is attached in https://github.com/NWUzhouwei/MPVConv. This work is supported by Shaanxi Province Key Research and Development Projects (No. 2020KW-068), General Project of Education Department of Shaanxi Provincial Government under Grant (No. 22JK058, No. 21JK0926), National Natural Science Foundation of China under Grant (No. 62106199, No. 62002290, No. 62001385), China Postdoctoral Science Foundation (No. 2021MD703883) and Jiangxi Provincial Natural Science Foundation under Grants (No. 20224BAB204051). 2023-11-22T01:59:47Z 2023-11-22T01:59:47Z 2023 Journal Article Zhou, W., Zhang, X., Hao, X., Wang, D. & He, Y. (2023). Multi point-voxel convolution (MPVConv) for deep learning on point clouds. Computers & Graphics, 112, 72-80. https://dx.doi.org/10.1016/j.cag.2023.03.008 0097-8493 https://hdl.handle.net/10356/172090 10.1016/j.cag.2023.03.008 2-s2.0-85151524236 112 72 80 en Computers & Graphics © 2023 Elsevier Ltd. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
3D Deep Learning
Point Clouds
spellingShingle Engineering::Computer science and engineering
3D Deep Learning
Point Clouds
Zhou, Wei
Zhang, Xiaodan
Hao, Xingxing
Wang, Dekui
He, Ying
Multi point-voxel convolution (MPVConv) for deep learning on point clouds
description The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN and DRINet, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to 36%, and achieves higher accuracy than the voxel-based model with up to 34× speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to 8× speedups. Also, our MPVConv only needs 65% of the GPU memory required by the latest point-voxel-based model (DRINet). The source code of our method is attached in https://github.com/NWUzhouwei/MPVConv.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Zhou, Wei
Zhang, Xiaodan
Hao, Xingxing
Wang, Dekui
He, Ying
format Article
author Zhou, Wei
Zhang, Xiaodan
Hao, Xingxing
Wang, Dekui
He, Ying
author_sort Zhou, Wei
title Multi point-voxel convolution (MPVConv) for deep learning on point clouds
title_short Multi point-voxel convolution (MPVConv) for deep learning on point clouds
title_full Multi point-voxel convolution (MPVConv) for deep learning on point clouds
title_fullStr Multi point-voxel convolution (MPVConv) for deep learning on point clouds
title_full_unstemmed Multi point-voxel convolution (MPVConv) for deep learning on point clouds
title_sort multi point-voxel convolution (mpvconv) for deep learning on point clouds
publishDate 2023
url https://hdl.handle.net/10356/172090
_version_ 1783955526771539968