Face super-resolution with large pose variation

• Main Author: Lim, Sheng Zhe
• Other Authors: Chen Change Loy
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/156468
• Institution: Nanyang Technological University

Recently many face super-resolution (FSR) algorithms have achieved great progress. The current interest lies in utilizing natural image priors from state of the art pre-trained Generative Adversarial Networks (GAN) to improve the restoration quality. With these pre-trained GAN priors, modern FSR works can recover rich texture and fine detail super-resolution (SR) face output from low resolution (LR) face input. However, while most FSR researchers focus on frontal or semi-frontal face super-resolution, the FSR performance on large face pose always be left out. To address this situation, I performed an analysis on a novel FSR algorithm - Generative Latent Bank (GLEAN) and found out that the performance of GLEAN face declined when the LR face input has a large face pose. Furthermore, through fine-tuning GLEAN with large face pose training data, I discovered GLEAN is able to learn and recover facial detail and texture of large face pose despite having GAN priors that are mostly trained on frontal faces. On top of that, I proposed a new cropped face dataset that contains over 200k+ images for evaluating FSR performance to encourage the community to focus on a small scale, extreme pose and heavily corrupted old face super-resolution problems. Finally, I concluded with future improvement directions on the proposed fine-tuned GLEAN.