Multi-stream spatiotemporal networks for driver fatigue detection from infrared and depth videos

• Main Author: Ma, Xiaoxi
• Other Authors: Yap Kim Hui
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/164796
• Institution: Nanyang Technological University

This thesis presents new methods for incorporating multi-stream networks into the driver fatigue detection system. For the depth video-based method, a two-stream CNN architecture is proposed to incorporate spatial information of the current depth frame and temporal information of neighboring depth frames which is represented by motion vectors. For the infrared video-based method, a convolutional three-stream network is proposed to incorporate current-infrared-frame-based spatial information, optical-flow-based short-term temporal information of two consecutive infrared frames, and optical flow-motion history image-based temporal information within the infrared video sequence.