Granular3D: Delving into multi-granularity 3D scene graph prediction
This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intr...
Saved in:
Main Authors: | , , , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2024
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/8811 https://ink.library.smu.edu.sg/context/sis_research/article/9814/viewcontent/Granular3D_av.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-9814 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-98142024-05-30T07:38:05Z Granular3D: Delving into multi-granularity 3D scene graph prediction HUANG, Kaixiang YANG, Jingru WANG, Jin HE, Shengfeng WANG, Zhan HE, Haiyan ZHANG, Qifeng LU, Guodong This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby comprehensively covering the contextual environments of instances. Moreover, Granular3D incorporates a Hierarchical Dual-Stage Network (HDSN), which differentiates and processes features of instances and their pairs at varying scales, leading to a targeted prediction of instance categories and their relationships. To advance the perception of sub-scene in HDSN, we design a Gather Point Transformer structure (GaPT) that enables the combinatorial interaction of local information from multiple point cloud sets, achieving a more comprehensive local contextual feature extraction. Extensive evaluations on the challenging 3DSSG benchmark demonstrate that our methods provide substantial improvements, establishing a new state-of-the-art in 3DSSG prediction, boosting the top-50 triplet accuracy by +2.8%. 2024-09-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8811 info:doi/10.1016/j.patcog.2024.110562 https://ink.library.smu.edu.sg/context/sis_research/article/9814/viewcontent/Granular3D_av.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University 3D point cloud 3D semantic scene graph prediction Gather point transformer Multi-granularity Graphics and Human Computer Interfaces Software Engineering |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
3D point cloud 3D semantic scene graph prediction Gather point transformer Multi-granularity Graphics and Human Computer Interfaces Software Engineering |
spellingShingle |
3D point cloud 3D semantic scene graph prediction Gather point transformer Multi-granularity Graphics and Human Computer Interfaces Software Engineering HUANG, Kaixiang YANG, Jingru WANG, Jin HE, Shengfeng WANG, Zhan HE, Haiyan ZHANG, Qifeng LU, Guodong Granular3D: Delving into multi-granularity 3D scene graph prediction |
description |
This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby comprehensively covering the contextual environments of instances. Moreover, Granular3D incorporates a Hierarchical Dual-Stage Network (HDSN), which differentiates and processes features of instances and their pairs at varying scales, leading to a targeted prediction of instance categories and their relationships. To advance the perception of sub-scene in HDSN, we design a Gather Point Transformer structure (GaPT) that enables the combinatorial interaction of local information from multiple point cloud sets, achieving a more comprehensive local contextual feature extraction. Extensive evaluations on the challenging 3DSSG benchmark demonstrate that our methods provide substantial improvements, establishing a new state-of-the-art in 3DSSG prediction, boosting the top-50 triplet accuracy by +2.8%. |
format |
text |
author |
HUANG, Kaixiang YANG, Jingru WANG, Jin HE, Shengfeng WANG, Zhan HE, Haiyan ZHANG, Qifeng LU, Guodong |
author_facet |
HUANG, Kaixiang YANG, Jingru WANG, Jin HE, Shengfeng WANG, Zhan HE, Haiyan ZHANG, Qifeng LU, Guodong |
author_sort |
HUANG, Kaixiang |
title |
Granular3D: Delving into multi-granularity 3D scene graph prediction |
title_short |
Granular3D: Delving into multi-granularity 3D scene graph prediction |
title_full |
Granular3D: Delving into multi-granularity 3D scene graph prediction |
title_fullStr |
Granular3D: Delving into multi-granularity 3D scene graph prediction |
title_full_unstemmed |
Granular3D: Delving into multi-granularity 3D scene graph prediction |
title_sort |
granular3d: delving into multi-granularity 3d scene graph prediction |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2024 |
url |
https://ink.library.smu.edu.sg/sis_research/8811 https://ink.library.smu.edu.sg/context/sis_research/article/9814/viewcontent/Granular3D_av.pdf |
_version_ |
1814047563806932992 |