Deep generative model for remote sensing

• Main Author: Huang, Shiqi
• Other Authors: Wen Bihan
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/167133
• Institution: Nanyang Technological University

Synthetic Aperture Radar (SAR) sensors are frequently used for earth monitoring in remote sensing. As SAR sensors can provide robust imagery for earth observation, researchers frequently attempt to apply conventional computer vision techniques to these images to improve monitoring and eliminate the need for manual processing. However, the lack of sufficient SAR images makes it difficult to train deep learning models which limits progress for traditional computer vision tasks in remote sensing. To address this issue, there has been growing interest in using Generative Adversarial Networks (GANs) to generate artificial data for data augmentation. In this project, we aim to use GANs to synthesise SAR images from available optical images. As it is common for real-world scenarios to have paired but not properly aligned images in a dataset, we propose integrating registration networks into GANs to address this misalignment issue. Our results show that the registration network effectively models the noise distribution in the dataset and improves performance compared to models without registration networks. Additionally, to enhance the fidelity of the generated images, we suggest integrating boundary and depth information from LiDAR images into optical images and then performing image translation. Our study demonstrates that this approach can produce more accurate and realistic SAR images than that without LiDAR images. This method has the potential to benefit future research in multi-modal image-to-image translation and remote sensing tasks and is also more cost-effective than traditional data acquisition methods. Finally, we provide an analysis of the proposed method, including its mechanism, advantages, and limitations, to guide future research in this area.