Noise matters in image restoration: from synthetic degradation to real-world challenges

• Main Author: Guo, Lanqing
• Other Authors: Wen Bihan
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Image restoration
• Online Access: https://hdl.handle.net/10356/174173
• Institution: Nanyang Technological University

Image restoration (IR) plays a crucial role in recovering high-quality image content from degraded observations, finding applications in surveillance, computational photography, and medical imaging. Noise is a common and fundamental degradation encountered in many IR scenarios. Despite its wide-ranging application in computer vision tasks, addressing noise is highly challenging due to its ill-posed nature, necessitating additional priors to make this problem tractable. Moreover, existing IR methods have demonstrated promising performance in certain ideal scenarios, assuming strict degradation conditions while often neglecting hybrid degradations. However, real-world scenarios encompass highly complicated degradations, such as noise combined with low illumination. This thesis focuses on exploring noise matters in these complex degradation scenarios, ranging from noise with specific distribution to real-world complicated noise. We first propose a novel method by exploring the non-local similarity in classical IR algorithms to achieve a highly efficient and effective Gaussian noise removal. Moving beyond specific Gaussian noise degradation, we extend our exploration to encompass robustness against different types of noise through joint image and noise modeling. To thoroughly investigate the practical applications, we then focus on the removal of real-world noise hidden in darkness and simultaneously enlightening the low-lightness with the introduced brightness-content disentanglement. After that, we extend the global darkness degradation to a partial one, i.e., shadow removal, where the shadow images have highly non-uniform illumination and noise degradation. In the third work, we exploit the non-shadow regions as contextual information, aiding in the restoration of shadow regions through a customized transformer network tailored to handle partial and non-uniform degradation. Finally, we introduce a diffusion-based method that jointly generates a shadow-free image and refines incorrectly detected shadow masks, effectively addressing noisy annotation problems. The main contributions of this thesis are three folds: We thoroughly study the noise matters that occurred in various real-world degradation scenarios; we propose various approaches to solve different noise matters image restoration problems from different angles; to verify the effectiveness of the proposed methods, we conduct extensive experiments on multiple datasets. The experimental results show that our methods outperform prior methods. The efforts and achievements presented in this thesis prompt the practical capabilities of noise matters image restoration techniques and provide fundamental support for future research.