Study of land-use and land-cover with medium resolution satellite imagery and super resolution technique

• Main Author: Chua, Wen Qing
• Other Authors: Lee Bu Sung, Francis
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/163167
• Institution: Nanyang Technological University

Remote sensing is the process of acquiring and analyzing spatially explicit information about the earth's geographical characteristics. With the launch of Sentinel-2 mission and its freely available satellite image datasets, new opportunities have risen in the fields of land-use mapping and environmental study. The combination of high resolution, novel spectral capabilities and short revisit cycle (5 days) will provide exceptional updated views of Earth crucial in land-use mapping. However, Sentinel-2 provides satellite images with a spatial resolution of 10m, which does not contain sufficient spatial details for accurate land-use classification. The purpose of this study is to enhance the spatial resolution of Sentinel-2 satellite images using deep learning, namely Generative Adversarial Networks (GAN) in the field of computer vision. Since Singapore has been experiencing rapid urbanisation over the past few years, this paper also further investigates other factors, such as data quality, that may affect the performance of the GAN models, with the aim to improve the super-resolution effects on Sentinel-2 images with urban landscapes characteristics. There are three main components to this paper. 1. Data preprocessing and exploratory analysis of Sentinel-2 data. 2. Design methodology and implementation of the DKN-SRGAN model 3. DKN-SRGAN experiments and evaluation. In the first component, data preprocessing methods such as cloud removal, image padding and splitting, and data augmentation, were conducted to produce datasets for GAN model training. To investigate the effect of data quality on model performance for images with urban landscapes characteristics like the ones in Singapore, images from selected cities of different landscape characteristics were also added. In the second component, we explore how our model, named DKN-SRGAN, estimates degradation kernels using deep learning and subsequently conducts noise injection to generate a dataset of near-natural low-resolution-high-resolution (LR-HR) image pairs. The low-resolution (LR) images were generated from the downsampling of the source images while the high-resolution (HR) image will be the source images themselves, with no prior need for other high-resolution images as reference. The SRGAN in DKN-SRGAN uses the generator in Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) as well as the discriminator in PatchGAN. On top of that, VGG-19 perceptual feature extractor is also used to extract features and visually enhance edges, so as to obtain super-resolution images with more precise details and improved perceptual quality. In the third component, several experiments were carried out to investigate factors affecting model performance. Evaluation of the experiments were performed with the quantitative comparison of the non-reference image quality assessment (NR-IQA) metrics like NIQE, PIQE, and BRISQUE, as well as visual comparisons of the generated images. Experimental findings were then used to further fine-tune this model to obtain more desirable resolution-enhancing of Sentinel-2 images with urban landscapes characteristics. The performance of our model was also evaluated with other commonly used image enhancement techniques such as the image sharpening function in OpenCV library. Overall, this study has shown that our proposed model has shown promising resolution-enhancing performance for Sentinel-2 images with urban landscapes characteristics. Based on the quantitative comparison of image quality assessment metrics and the comparison of visual effects, our proposed DKN-SRGAN model outperforms commonly used image enhancement techniques.