Cloud removal in optical remote sensing imagery based on multimodality image reconstruction using deep learning

• Main Author: Zhan, Hang
• Other Authors: Lu Yilong
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/154878
• Institution: Nanyang Technological University

It is widely known that optical satellite image is vulnerable to cloud contamination. Our task is to solve the problem of cloud removal by image blending based on the provided high-resolution composite images. The wide spread of machine learning, especially deep learning (DL), makes our problem solvable. We put forward a framework to leverage the advantages of Generative Adversarial Networks (GAN) and the traditional gradient-based method. Concretely, Gaussian-Poisson Equation is employed to formulate the problem of high-resolution image blending, which can be regarded as a joint optimization constrained by image color and gradient information. In our work, two deep learning models were built and tested. Firstly, to obtain the color constraint, we put forward a model named Blending GAN to learn the mapping between the composite images and the well-blended ones, which could generate lowresolution but realistic images. Secondly, we can obtain gradient constraint using the gradient filter. However, the discontinuity may exist in the gradient map due to inaccurate image composition. As we know, the Synthetic Aperture Radar (SAR) data is attracting equal attention from both industry and academia. One major advantage is its robustness against cloud contamination. Hence, we propose a Gradient Repair GAN and explore the use of SAR data to produce more accurate gradient maps, which can be better used for following processing. After integration of color with gradient information based on Gaussian-Poisson Equation, the ultimate high-resolution well-blended image, i.e. cloudless images could be obtained. The final performance is numerically evaluated based on PSNR and shows promising performance