Cloud removal in optical remote sensing imagery based on multimodality image fusion with machine learning

• Main Author: Chai, Dong
• Other Authors: Lu Yilong
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/157272
• Institution: Nanyang Technological University

Satellite optical images are widely used in updating landform. However, it is a common issue that a portion of satellite optical images may be covered by clouds. Besides, the Synthetic Aperture Radar (SAR) imageries are not influenced by clouds and provide topographic information. With the resolution of SAR data improving rapidly through recent years, using SAR imageries to enhance data from other sources could be a more promising topic for research as well as practical applications. The task is to remove the cloud from Satellite optical images with the help of Synthetic Aperture Radar (SAR) imageries by using deep learning (DL) models. Specifically, in this work a framework to leverage the advantages of Generative Adversarial Networks (GAN) and the traditional gradient-based method is put forward. Concretely, Gaussian- Poisson Equation is employed to formulate the problem of high-resolution image blending, which can be regarded as a joint optimization constrained by image color and gradient information. In this work, we divide the whole task into two parts and developed two DL models respectively. Firstly, to obtain the color constraint, we put forward a Generative Adversarial Networks (GANs) Blending model to learn the mapping between the composite images with clouds and the well-blended ones. After applying this model, we could generate low- resolution but realistic images. Secondly, we can obtain gradient constraint by using the gradient filter. However, the discontinuity may exist in the gradient map due to inaccurate image composition. Hence, we propose a another Gradient Repair GAN model and explore the use of SAR data to produce more accurate gradient maps. After integration of color with gradient information based on Gaussian-Poisson Equation, the ultimate high-resolution well-blended image, i.e. cloudless images could be obtained. The final performance is numerically evaluated based on PSNR and shows promising performance.