MASKED FACE RECOGNITION USING DEEP LEARNING

• Main Author: Firdaus, Fauzan
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/63137
• Institution: Institut Teknologi Bandung

ABSTRACT MASKED FACE RECOGNITION USING DEEP LEARNING By Fauzan Firdaus NIM: 23520011 (Master’s Program in Informatics) The end of 2019 is an unforgettable time for all of humanity. At that time, the first case of COVID-19 (Corona Virus Disease) appeared in China. Starting from there, this virus spread quickly to various regions of the world. Until around the beginning of 2020, the first case appeared in Indonesia. The existence of COVID-19 in this world has a very massive impact on human life. Every aspect such as economic, social, academic, and human lifestyle are directly affected by the existence of this pandemic. One of the most visible effect and fast-paced lifestyle changes is the use of a face mask. Although face masks are not 100% protective from contracting COVID-19, at least the use of a face mask is believed to be able to inhibit or minimize the spread of COVID-19 itself. The use of a face mask indirectly affects the performance of an existing face recognition system. Based on existing studies, the face recognition system already has a test accuracy performance of close to 100%. On the other hand, research conducted by Mundial et al. (2020) revealed that the use of face masks achieved a test accuracy below 80% with the same training technique. Although the other test accuracy was 97%, the training data they used contained masked faces. Therefore, in this new situation, a new mechanism is needed in the development of a masked face recognition system in order to have better performance, with the challenge of training data that does not contain any masked face data. By wearing a face mask, it means that there are missing features or information in a face. The remaining facial features are available only around the forehead and eyes. Research conducted by Elmahmudi et al. (2018 and 2019) revealed that the recognition rate of the top half of the face had a high score compared to some other parts. Based on this statement, the concept of using data (for training data, validation, and test data) in this study will focus on the part of the face that is not covered by the mask (top half of the face) only. This mechanism applies the concept of human vision when recognizing someone who is known when they wearing a face mask. When we recognize people we knew but are in the condition of wearing a face mask, then we will not consider their masks as their identity. We will recognize them based on other parts such as their forehead, eyebrows, eyes, or maybe their hair. iv The open-source dataset of masked faces is currently limited. Therefore, the dataset for this thesis research will be built independently. The number of subjects or individuals who have been recorded has been collected up to 125 people. The dataset format is a video with a duration of five to seven seconds taken from the left to the right side of the corresponding person, with two videos for each person (one for a masked face, another one for an unmasked face). After each video is converted into a frame or image, the image will be further preprocessed namely face detection and other processes with the aim of perfecting the input data for the feature extraction stage. YOLOv4 is used as a face detection method in this thesis research. The feature extraction method used in this thesis research is similar to the reference research (Elmahmudi et al. 2018 and 2019), namely VGG-Face (pre-train model). In addition, as a comparison, VGG-16 (non-pre-train model which is the basic architecture of VGG-Face) will be used and ANN (Artificial Neural Network) is used as classifiers. For VGG-Face, ANN and CS (Cosine Similarity) are used for the classifier. Based on the experimental results, the best performance was obtained by the VGG-Face feature extraction method and the ANN classifier with a test accuracy of 99.57%. On the other hand, the performance of the previous training technique (Mundial et al. 2020) obtained a test accuracy of 79.58% with the same dataset and method. This proves that the proposed training technique has a very large impact on the performance of the masked facial recognition system that has been built. Keywords: Masked Face, Face Recognition, Deep Learning, Unmasked Area Training