Facial expression transfer using StyleGAN

• Main Author: Zhao, Lan
• Other Authors: Tan Yap Peng
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/167195
• Institution: Nanyang Technological University

Facial expression transfer is a hot topic in computer vision and computer graphics. Given a source image and a target expression, the goal is to transfer the target expression to the source image. StyleGAN, a notable improvement in the development of Generative Adversarial Networks (GANs), is a powerful tool to generate high-fidelity and high-resolution images of human faces. This project designs a pipeline to perform facial expression transfer with customized StyleGAN and Pixel2style2pixel (pSp) models. Specifically, pSp encoder is used to obtain the latent representation (latent code) of the source image, which is afterwards manipulated based on the target expression. StyleGAN generator is then used to generate the resulting image from the manipulated latent code. This project also explores the latent space of StyleGAN, analyses problems like model generalizability and expressiveness-editability trade-off. Finally, a pipeline is successfully built for both image and video-based facial expression transfer. Experiment results demonstrate the effectiveness of the proposed method.