Deepfacade: A deep learning approach to facade parsing

• Main Authors: LIU, Hantang, ZHANG, Jialiang, ZHU, Jianke, HOI, Steven C. H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Artificial intelligence; Deep neural networks; Facades; Formal languages; Image segmentation; Neural networks; Semantics; Building facades; Convolutional neural network; Learning approach; Learning-based methods; Man-made structures; Segmentation results; Semantic category; State-of-the-art methods; Deep learning; Artificial Intelligence and Robotics; Databases and Information Systems; Numerical Analysis and Scientific Computing
• Online Access: https://ink.library.smu.edu.sg/sis_research/3849; https://ink.library.smu.edu.sg/context/sis_research/article/4851/viewcontent/0320.pdf
• Institution: Singapore Management University

The parsing of building facades is a key component to the problem of 3D street scenes reconstruction, which is long desired in computer vision. In this paper, we propose a deep learning based method for segmenting a facade into semantic categories. Man-made structures often present the characteristic of symmetry. Based on this observation, we propose a symmetric regularizer for training the neural network. Our proposed method can make use of both the power of deep neural networks and the structure of man-made architectures. We also propose a method to refine the segmentation results using bounding boxes generated by the Region Proposal Network. We test our method by training a FCN-8s network with the novel loss function. Experimental results show that our method has outperformed previous state-of-the-art methods significantly on both the ECP dataset and the eTRIMS dataset. As far as we know, we are the first to employ end-to-end deepconvolutional neural network on full image scale in the task of building facades parsing.