Face tracking for people with or without wearing face masks

• Main Author: Teh, Boon Jie
• Other Authors: Lin Zhiping
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/149315
• Institution: Nanyang Technological University

Face Tracking (FT) algorithm serves as the backbone of modern public security tool. FT is a two-step process where faces are detected each frame, followed by associations of detections across frames in a video sequence. The core that influences accuracy of the algorithm in tracking human is the accuracy of face detection process and is therefore the focus of this Final Year Project. Many state-of-the-art deep neural network algorithms are much less accurate in detecting human that wear masks as only limited number of images, in the range of hundreds, with human wearing masks are available. Thus, the performances of these algorithms are questioned during the COVID-19 pandemic as wearing face masks has been prevalent. Although existing virtual mask augmentation program [4] on public benchmark datasets could allow quick creation of synthetic dataset with human wearing masks, it only augments sharp face mask regardless of the image quality of faces in images. The aim of this Final Year Project is thus to make several refinements to 1) improve accuracy and robustness of face detection and face alignment algorithms in [4] and 2) measure face sharpness of all faces to blur virtual mask before the creation of synthetic dataset. In-plane rotation of human faces are made for challenging head pose situations. A better face alignment model that uses a 3D face reconstruction method is utilized to substitute the original, less robust model. Image pre-processing is also carried out to improve the robustness of face sharpness measurement. Being inspired by the 5 ordinal face sharpness categories in [6], 5 different mask blurring strength is created to blur face mask according to the face sharpness measurements.