Classification of deepfakes and ‘beautified’ media

• Main Author: Addi, Debashree
• Other Authors: Liu Yang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/148049
• Institution: Nanyang Technological University

Recently, videos and images have known to be manipulated using deep learning to generate fake videos and images which appear to be real, widely known as deepfakes. These deepfakes are a cause for high concern as they tend to disrupt the integrity of videos in the form of financial frauds, hoaxes and fake news. A lot of technologies have been developed to detect a video’s legitimacy, with relatively high accuracy. However, with upcoming increasingly intelligent AI systems to generate deepfakes, technology has proven to be inadequate to detect them. Along with deepfakes, another category, known as ‘beautified’ media, has also arisen as a result of digital manipulation. This media consists of images or videos that have been processed through addition of filters, makeup or smoothened skin. This ‘beautified’ media may be mistakenly classified as a deepfake, by deepfake detector algorithms whereas they are just real videos and images with an added filter. This creates a need to develop a model to accurately differentiate between ‘real’, ‘fake’ and ‘beautified’ media. In this study, the Keras-Xception model, which provides high accuracy, was used to attempt to classify digital media. The model was trained on the FaceForensics++ dataset, which comprises of both original and manipulated YouTube video sequences. The dataset was additionally augmented with ‘beautified’ versions of the same data, using the BeautyCamera application, to facilitate classification into the three main categories – ‘real’, ‘fake’ and ‘beautified’. Multiple Xception models were trained on different kinds of preprocessing of the FaceForensics++ dataset in order to make model comparisons and declare the best performing model. Finally, the classifier was developed with a high model accuracy of 89 percent and further tested on unseen data. Keywords - Deepfake detector, ‘beautified’ media, Keras-Xception model, BeautyCamera application