Facial micro-expression analysis

• Main Author: Ong, Si Ying
• Other Authors: Zheng Jianmin
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/165884
• Institution: Nanyang Technological University

Individuals' emotional states are revealed through their facial expressions. Nevertheless, emotions can be concealed and portrayed differently, making discerning an individual's genuine emotion challenging. Additionally, in medical settings, individuals may be reluctant to discuss their genuine emotions and condition during a remote consultation, which reduces the accuracy of the medical diagnosis. Patients may be reluctant to address particular problems or hesitant to seek help. As a result, the utilization of facial micro-expression analysis technologies enables the decoding of deliberately disguised subtle emotions. Therefore, the research intends to employ such technology to enable healthcare professionals to interpret a patient's genuine emotions and administer appropriate medical care or consolation, if necessary, facilitating accurate medical diagnosis and proper treatment. The report proposed developing a real-time mobile application that incorporates neural network models into tele-consultation settings in an Android mobile application that accurately recognizes facial micro-expressions. To achieve a reputable performance for the system, the model employs Advanced Attention Network (AAN) inspired by neural network models Distract Your Attention Network (DAN) and Convolutional Block Attention Module (CBAM), as well as a series of experiments with different configurations and hyperparameters, was conducted. Based on the results, the AAN model performance was inadequate and the research implemented more trials with various configurations and model fine-tuning. Further experiment implies that the positioning of the attention mechanism, CBAM, and the hyperparameters and architecture of neural network models influence their performance, resulting in improved accuracy. The research implications imply that more experiments can be conducted to examine the effects of adjusting the position of CBAM and other hyperparameters, which may result in greater accuracy in identifying "genuine" emotion during tele-consultation.