Single image reflection removal with camera calibration

• Main Author: Chen, Jinnan
• Other Authors: Jiang Xudong
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/150528
• Institution: Nanyang Technological University

With the rapid development of deep learning algorithms and powerful computing hardware, some classic ill-posed computer vision problems could be better solved in a data-driven manner. For single image reflection removal, researchers use different prior knowledge to make the loss design more reasonable. My research on this topic is based on the prior knowledge of physical modeling of the reflection formation. I integrated my research work in NTU as a master by research student into this final thesis, with the title of Stationary Single Image Reflection Removal: Physical Rule to Robustness. This thesis mainly aims to analyze and give a reliable answer to ‘how to encode the physical information into the designed network thus making it easily generalized in the real world and robust to some input perturbations’, before which, some new ideas and in-depth investigation about the robustness of CNN-based model in terms of adversarial perturbations are introduced. The main part of the thesis consists of two specific physical-rule-based models to simulate the reflection formation process. The first one focuses on how to use the pixel-wise reflection coefficient map to guide the reflection removal process as well as calibrate the camera parameters while the second one focuses on how to add constraints to stabilize the output in terms of the perturbations from some physical parameters of the glass which have a great effect to the reflection part. Substantial experimental results and analysis show the effectiveness and the superiority of the proposed methods in the different real scenarios and this research could have a broad impact on other computer vision problems.