Audio spectrogram deception detection

• Main Author: Gao, Ziqi
• Other Authors: Alex Chichung Kot
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering::Electronic systems::Signal processing
• Online Access: https://hdl.handle.net/10356/170278
• Institution: Nanyang Technological University

Automatic deception detection is an important research direction, and traditional detection methods based on physiological signals are difficult to be implemented in real life. Video-based detection methods are a better alternative. However, each culture has its own way of expressing deception, and among all the de- ception detection studies, there are more studies for Western cultures. In this project, an Asian ethnic deception detection dataset will be collected. The au- dio spectrograms generated from the videos will be applied to different trans- former models (ViT, DeiT, VPT) for deception detection studies. We will per- form experiments on three public datasets (BoL, Mu3d, RLT) and self-collected datasets. The results of the different models will be compared, showing that DeiT is the most suitable model for automatic deception detection among the three for spectrograms. The results of the self-collected dataset were also com- pared to the public dataset, showing that the self-collected data showed an im- provement of 1.74% and 5.28% for the BoL and Mu3d public datasets, respec- tively.