Gesture recognition in car cabin environment

• Main Author: He, Siqi
• Other Authors: Yap Kim Hui
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Gesture recognition; 3D convolutional neural networks; Res-C3D network; LSTM; Attention mechanisms
• Online Access: https://hdl.handle.net/10356/181406
• Institution: Nanyang Technological University

As a research hotspot in the field of computer vision, gesture recognition has received extensive attention in recent years. With the rapid development of intelligent driving technology, in-car human-computer interaction has gradually become an important research direction. Gesture recognition shows a wide range of application prospects because of its natural and intuitive interaction mode. This dissertation aims to implement and optimize in-car gesture recognition technology and compare the performance of different deep learning models based on the NVIDIA Gesture Dynamic Hand Gesture (NVGesture) Dataset. After preprocessing the video dataset, I implemented and evaluated the basic 3DCNN model, ResNet model, ResNet model with attention, CNN-LSTM model, and CNN-LSTM model with attention for in-depth analysis of their performance in gesture recognition tasks. The experimental results show that the 3DCNN model achieved a lower accuracy of approximately 78.97% during the gesture recognition tasks. Then, the structure of ResNet models, particularly with attention mechanisms, showed slight improvements in performance. Ultimately, the CNN-LSTM model, especially the version enhanced with attention mechanisms, demonstrated the highest accuracy, reaching 95.14%. The research shows that gesture recognition technology based on the NVGesture dataset has great application potential in the intelligent interaction system in the car, and can be further applied in the driver assistance system and intelligent cockpit in the future.