Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds

Point clouds provide intrinsic geometric information and surface context for scene understanding. Existing methods for point cloud segmentation require a large amount of fully labeled data. Using advanced depth sensors, collection of large scale 3D dataset is no longer a cumbersome process. However,...

Full description

Saved in:
Bibliographic Details
Main Authors: Wei, Jiacheng, Lin, Guosheng, Yap, Kim-Hui, Hung, Tzu-Yi, Xie, Lihua
Other Authors: School of Computer Science and Engineering
Format: Conference or Workshop Item
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/144256
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-144256
record_format dspace
spelling sg-ntu-dr.10356-1442562020-10-23T06:31:31Z Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds Wei, Jiacheng Lin, Guosheng Yap, Kim-Hui Hung, Tzu-Yi Xie, Lihua School of Computer Science and Engineering 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Engineering::Computer science and engineering Three-dimensional Displays Data Mining Point clouds provide intrinsic geometric information and surface context for scene understanding. Existing methods for point cloud segmentation require a large amount of fully labeled data. Using advanced depth sensors, collection of large scale 3D dataset is no longer a cumbersome process. However, manually producing point-level label on the large scale dataset is time and labor-intensive. In this paper, we propose a weakly supervised approach to predict point-level results using weak labels on 3D point clouds. We introduce our multi-path region mining module to generate pseudo point-level labels from a classification network trained with weak labels. It mines the localization cues for each class from various aspects of the network feature using different attention modules. Then, we use the point-level pseudo label to train a point cloud segmentation network in a fully supervised manner. To the best of our knowledge, this is the first method that uses cloud-level weak labels on raw 3D space to train a point cloud semantic segmentation network. In our setting, the 3D weak labels only indicate the classes that appeared in our input sample. We discuss both scene- and subcloud-level weakly labels on raw 3D point cloud data and perform in-depth experiments on them. On ScanNet dataset, our result trained with subcloud-level labels is compatible with some fully supervised methods. AI Singapore Ministry of Education (MOE) National Research Foundation (NRF) Accepted version This work was conducted within the Delta-NTU Corporate Lab for Cyber-Physical Systems with funding support from Delta Electronics Inc. and the National Research Foundation (NRF) Singapore under the Corp Lab@University Scheme. This work is also partly supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2018-003) and the MOE Tier-1 research grant: RG22/19 (S). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. 2020-10-23T06:31:30Z 2020-10-23T06:31:30Z 2020 Conference Paper Wei, J., Lin, G., Yap, K.-H., Hung, T.-Y., & Xie, L. (2020). Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds. Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR42600.2020.00444 https://hdl.handle.net/10356/144256 10.1109/CVPR42600.2020.00444 en AISG-RP-2018-003 RG22/19 (S) © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/CVPR42600.2020.00444 application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Three-dimensional Displays
Data Mining
spellingShingle Engineering::Computer science and engineering
Three-dimensional Displays
Data Mining
Wei, Jiacheng
Lin, Guosheng
Yap, Kim-Hui
Hung, Tzu-Yi
Xie, Lihua
Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
description Point clouds provide intrinsic geometric information and surface context for scene understanding. Existing methods for point cloud segmentation require a large amount of fully labeled data. Using advanced depth sensors, collection of large scale 3D dataset is no longer a cumbersome process. However, manually producing point-level label on the large scale dataset is time and labor-intensive. In this paper, we propose a weakly supervised approach to predict point-level results using weak labels on 3D point clouds. We introduce our multi-path region mining module to generate pseudo point-level labels from a classification network trained with weak labels. It mines the localization cues for each class from various aspects of the network feature using different attention modules. Then, we use the point-level pseudo label to train a point cloud segmentation network in a fully supervised manner. To the best of our knowledge, this is the first method that uses cloud-level weak labels on raw 3D space to train a point cloud semantic segmentation network. In our setting, the 3D weak labels only indicate the classes that appeared in our input sample. We discuss both scene- and subcloud-level weakly labels on raw 3D point cloud data and perform in-depth experiments on them. On ScanNet dataset, our result trained with subcloud-level labels is compatible with some fully supervised methods.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Wei, Jiacheng
Lin, Guosheng
Yap, Kim-Hui
Hung, Tzu-Yi
Xie, Lihua
format Conference or Workshop Item
author Wei, Jiacheng
Lin, Guosheng
Yap, Kim-Hui
Hung, Tzu-Yi
Xie, Lihua
author_sort Wei, Jiacheng
title Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
title_short Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
title_full Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
title_fullStr Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
title_full_unstemmed Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds
title_sort multi-path region mining for weakly supervised 3d semantic segmentation on point clouds
publishDate 2020
url https://hdl.handle.net/10356/144256
_version_ 1683492956967796736