Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation

In recent years, significant progress has been made in semantic segmentation and 3D geometric mapping. As a result, multi-robot systems are expected to operate in increasingly complex environment with intelligent ability, such as dynamic perception and active navigation. In this case, comprehensive...

Full description

Saved in:
Bibliographic Details
Main Author: Li, Ruilin
Other Authors: Wang Dan Wei
Format: Theses and Dissertations
Language:English
Published: 2019
Subjects:
Online Access:http://hdl.handle.net/10356/78493
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-78493
record_format dspace
spelling sg-ntu-dr.10356-784932023-07-04T15:55:37Z Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation Li, Ruilin Wang Dan Wei School of Electrical and Electronic Engineering DRNTU::Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics In recent years, significant progress has been made in semantic segmentation and 3D geometric mapping. As a result, multi-robot systems are expected to operate in increasingly complex environment with intelligent ability, such as dynamic perception and active navigation. In this case, comprehensive contextual understanding of the surroundings is a critical challenge for multi-robot perception system. Attempts have been made from single robot semantic mapping, collaborative 3D geometric mapping. However, there is a gap in performing accurate and largescale semantic mapping for multiple robots. This thesis goes a step further by focusing on a hierarchical multi-robot semantic mapping framework. Prior to semantic mapping, the semantic segmentation model is added to the robot system so that the robot can perceive the semantic information of the surrounding environment in real time at a speed of 2 Hz. Then, the thesis proposes a novel hierarchical multi-robot semantic mapping framework, where the problem is addressed in low level single robot semantic mapping and high level global semantic mapping. In the single robot semantic mapping process, Bayesian rules are used for label fusion and occupancy probability update, where the semantic information is added to the geometric map grid. High level global semantic map fusion covers decentralized map sharing and global semantic map updating. Collaborative semantic reconstruction is conducted in two scenarios, that is, NTU dataset and the KITTI dataset. The results show the high quality of the global semantic map, which demonstrates the efficiency, accuracy and versatility of 3D semantic map fusion algorithm in multi-robot missions. Master of Science (Computer Control and Automation) 2019-06-20T08:46:06Z 2019-06-20T08:46:06Z 2019 Thesis http://hdl.handle.net/10356/78493 en 70 p. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic DRNTU::Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
spellingShingle DRNTU::Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
Li, Ruilin
Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
description In recent years, significant progress has been made in semantic segmentation and 3D geometric mapping. As a result, multi-robot systems are expected to operate in increasingly complex environment with intelligent ability, such as dynamic perception and active navigation. In this case, comprehensive contextual understanding of the surroundings is a critical challenge for multi-robot perception system. Attempts have been made from single robot semantic mapping, collaborative 3D geometric mapping. However, there is a gap in performing accurate and largescale semantic mapping for multiple robots. This thesis goes a step further by focusing on a hierarchical multi-robot semantic mapping framework. Prior to semantic mapping, the semantic segmentation model is added to the robot system so that the robot can perceive the semantic information of the surrounding environment in real time at a speed of 2 Hz. Then, the thesis proposes a novel hierarchical multi-robot semantic mapping framework, where the problem is addressed in low level single robot semantic mapping and high level global semantic mapping. In the single robot semantic mapping process, Bayesian rules are used for label fusion and occupancy probability update, where the semantic information is added to the geometric map grid. High level global semantic map fusion covers decentralized map sharing and global semantic map updating. Collaborative semantic reconstruction is conducted in two scenarios, that is, NTU dataset and the KITTI dataset. The results show the high quality of the global semantic map, which demonstrates the efficiency, accuracy and versatility of 3D semantic map fusion algorithm in multi-robot missions.
author2 Wang Dan Wei
author_facet Wang Dan Wei
Li, Ruilin
format Theses and Dissertations
author Li, Ruilin
author_sort Li, Ruilin
title Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
title_short Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
title_full Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
title_fullStr Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
title_full_unstemmed Hierarchical framework of collaborative 3D semantic mapping based on semantic segmentation
title_sort hierarchical framework of collaborative 3d semantic mapping based on semantic segmentation
publishDate 2019
url http://hdl.handle.net/10356/78493
_version_ 1772825623167762432