Depth map upsampling via multi-modal generative adversarial network

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. Autonomous robots for smart homes and smart cities mostly require depth perception in order to interact with their environments. However, depth maps are usually captured in a lower resolution as compared to RGB color images due to the inheren...

Full description

Saved in:
Bibliographic Details
Main Authors: Tan, Daniel Stanley, Lin, Jun Ming, Lai, Yu Chi, Ilao, Joel P., Hua, Kai Lung
Format: text
Published: Animo Repository 2019
Online Access:https://animorepository.dlsu.edu.ph/faculty_research/849
https://animorepository.dlsu.edu.ph/context/faculty_research/article/1848/type/native/viewcontent
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: De La Salle University
id oai:animorepository.dlsu.edu.ph:faculty_research-1848
record_format eprints
spelling oai:animorepository.dlsu.edu.ph:faculty_research-18482022-07-16T01:27:21Z Depth map upsampling via multi-modal generative adversarial network Tan, Daniel Stanley Lin, Jun Ming Lai, Yu Chi Ilao, Joel P. Hua, Kai Lung © 2019 by the authors. Licensee MDPI, Basel, Switzerland. Autonomous robots for smart homes and smart cities mostly require depth perception in order to interact with their environments. However, depth maps are usually captured in a lower resolution as compared to RGB color images due to the inherent limitations of the sensors. Naively increasing its resolution often leads to loss of sharpness and incorrect estimates, especially in the regions with depth discontinuities or depth boundaries. In this paper, we propose a novel Generative Adversarial Network (GAN)-based framework for depth map super-resolution that is able to preserve the smooth areas, as well as the sharp edges at the boundaries of the depth map. Our proposed model is trained on two different modalities, namely color images and depth maps. However, at test time, our model only requires the depth map in order to produce a higher resolution version. We evaluated our model both quantitatively and qualitatively, and our experiments show that our method performs better than existing state-of-the-art models. 2019-04-01T07:00:00Z text text/html https://animorepository.dlsu.edu.ph/faculty_research/849 https://animorepository.dlsu.edu.ph/context/faculty_research/article/1848/type/native/viewcontent Faculty Research Work Animo Repository
institution De La Salle University
building De La Salle University Library
continent Asia
country Philippines
Philippines
content_provider De La Salle University Library
collection DLSU Institutional Repository
description © 2019 by the authors. Licensee MDPI, Basel, Switzerland. Autonomous robots for smart homes and smart cities mostly require depth perception in order to interact with their environments. However, depth maps are usually captured in a lower resolution as compared to RGB color images due to the inherent limitations of the sensors. Naively increasing its resolution often leads to loss of sharpness and incorrect estimates, especially in the regions with depth discontinuities or depth boundaries. In this paper, we propose a novel Generative Adversarial Network (GAN)-based framework for depth map super-resolution that is able to preserve the smooth areas, as well as the sharp edges at the boundaries of the depth map. Our proposed model is trained on two different modalities, namely color images and depth maps. However, at test time, our model only requires the depth map in order to produce a higher resolution version. We evaluated our model both quantitatively and qualitatively, and our experiments show that our method performs better than existing state-of-the-art models.
format text
author Tan, Daniel Stanley
Lin, Jun Ming
Lai, Yu Chi
Ilao, Joel P.
Hua, Kai Lung
spellingShingle Tan, Daniel Stanley
Lin, Jun Ming
Lai, Yu Chi
Ilao, Joel P.
Hua, Kai Lung
Depth map upsampling via multi-modal generative adversarial network
author_facet Tan, Daniel Stanley
Lin, Jun Ming
Lai, Yu Chi
Ilao, Joel P.
Hua, Kai Lung
author_sort Tan, Daniel Stanley
title Depth map upsampling via multi-modal generative adversarial network
title_short Depth map upsampling via multi-modal generative adversarial network
title_full Depth map upsampling via multi-modal generative adversarial network
title_fullStr Depth map upsampling via multi-modal generative adversarial network
title_full_unstemmed Depth map upsampling via multi-modal generative adversarial network
title_sort depth map upsampling via multi-modal generative adversarial network
publisher Animo Repository
publishDate 2019
url https://animorepository.dlsu.edu.ph/faculty_research/849
https://animorepository.dlsu.edu.ph/context/faculty_research/article/1848/type/native/viewcontent
_version_ 1738854830479245312