GENERATIVE ADVERSARIAL NETWORK (GAN) BASED MAKEUP TRANSFER SYSTEM FOR VARIOUS SKIN TONE

• Main Author: Pradnya Pramudita, Raras
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/66326
• Institution: Institut Teknologi Bandung

Even though wearing makeup is a chore that women do every day, this activity may also come with a challenge. Wearing makeup needs certain skills because each face has unique shape, colour, and also size of facial features. The right makeup look will produce illusion that successfully highlight the best features and cover up the less attractive features of the face. Makeup also needs to be adjusted according the place, time, and clothing where the user will wear the makeup. Therefore, makeup users need to determine what makeup look suits their face and also the setting. Makeup transfer is a deep learning model that has an objective to transfer makeup style from a reference image with makeup face to another face that does not use makeup. This task is able to answer challenges that makeup users are currently facing. Unfortunately, most applications of makeup transfer models that have now been published can only be used optimally by people with light skin tone. This is due to the limitations of the dataset that has been collected and also framework of the model that does not support the inclusivity of the user's skin tone. The principal contribution of this this Final Thesis research is a generative adversarial network (GAN) based makeup transfer network that can transfer makeup from a reference image to original image which can receive various user’s and reference face’s skin tone. The objective of this model is to be able to capture the makeup of reference image, transfer the makeup to original image, while also preserve the identity of original images including their skin tone. Adjustments for ground truth system, model architecture, and loss functions are applied so that the model can be used by user with various skin tone. Changes to ground truth were made using warping and Poisson blending method. Furthermore, experiments were carried out on the GAN model that was used as a reference, namely BeautyGAN, by pairing two types of model architecture: ResNet and Dilated ResNet for the generator model. Changes to the ground truth and loss function effectively encourage the model to achieve desired inference results. Furthermore, both types of architectural models are able to produce good quality images. However, the proposed model using Dilated ResNet is able to produce better results objectively based on image quality and also subjectively based on facial makeup transfer quality.