Single image reflection separation through machine learning

• Main Author: Lu, Zhan
• Other Authors: Jiang Xudong
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering::Computer hardware, software and systems
• Online Access: https://hdl.handle.net/10356/141134
• Institution: Nanyang Technological University

Reflection separation is a traditional computer vision problem aiming at enhancing the visibility of the objects behind glass by separating the unwanted reflection part of the captured image. Using one single image to realize it is the most convenient and less-constrains approach. However, it is a massive challenge with its ill-posed nature that we try to separate transmission layer and reflection layer as two outputs from a single input. To make problem more treatable, many works struggle to solve it with some physical priors from observations, but all of them face the problem of unstable performance in different scenarios. With the development of machine learning, the learning-based method gives a new way to address this separation problem. We propose a separation network as our framework with the help of camera calibration prediction. Different from previous methods, we improve the blending mask for synthesis by imposing camera calibration cues as new constraints. Besides, we design an image synthesis method for training data solving the problem of shortage on real datasets, which follows physical rules to simulate the case as real as possible. Then, we collect a real dataset containing 320 samples with corresponding camera calibration information for evaluation. We demonstrate that our method achieves state-of-the-art performance by some comparisons with the other five newest methods. Furthermore, we develop a panorama estimation method based on the outputs from reflection separation network, which helps to alleviate the ill-posed problem of estimating panorama from a single image.