Deep learning for image and video understanding

• Main Author: Yue, Kunlun
• Other Authors: Tan Yap Peng
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computing methodologies; Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/139497
• Institution: Nanyang Technological University

Gaze detection is a sub-area under object detection and becomes more and more popular for its wide applications that are useful in our daily life. For example, the gaze following analysis can be quite useful in smart-study system to monitor the students’ studying situations. In this report, we focus on another topic under gaze analysis---Looking at each other(LAEO). Knowing whether peoples are LAEO can help us understand their relationships because mutual gaze between people is a very important non-verbal communication. Most of the methods presented and used focus on analyzing mutual gaze in an individual frame. But this report will talk about a new method, which will conduct this analysis in a spatio-temporal approach. Continual frames and videos will be used as input data. Then we will extract the heads to create tracks(a list of heads that heaped in accordance with time) and get the respective heads_map as inputs for the model. Finally the system will decide whether the peoples are LAEO by giving the probability(Given by LAEO score) of LAEO. The results on common meeting room videos demonstrate the effectiveness of the new method and model. Hopefully, this system can be used in the real-time applications to monitor or analyze people in a meeting room after some future works.