Towards interpretable & robust occluded facial recognition

• Main Author: Rachita, Agrawal
• Other Authors: Lin Weisi
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/166085
• Institution: Nanyang Technological University

With rapid technological advances, robust facial recognition systems have become necessary to strengthen security, and deep Convolutional Neural Networks are gaining popularity in enhancing such systems. However, facial recognition algorithms still face challenges when tested in real-world scenarios due to light and pose variations and the presence of facial occlusions. The main objective of this research is to improve the robustness of an existing reference facial recognition model, which utilizes feature masks to detect synthetic occlusions, and make it handle occlusions better in an unconstrained environment. We researched three fundamental approaches: feature dictionaries, Feature Pyramid Networks, and image segmentation. We achieved these methods using LResnet50E-IR, Transformers, Resnet50, and U-Net models. Post analysis, we found that the most effective solution was the U-Net model with LResnet50E-IR backbone, as it could accurately detect the boundary of the visible part of the face. Moreover, our model outperformed existing methods, especially with real-world occluded face images. Our experiments show that adopting such an approach can significantly improve the accuracy of modern facial recognition algorithms. Future work can be done to investigate deeper image segmentation models further and combine them with Feature Pyramid Networks to enhance facial recognition models.