AI-based indoor human gesture recognition using FMCW radar

• Main Author: Liu, Boya
• Other Authors: Wen Bihan
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/161668
• Institution: Nanyang Technological University

Human gesture recognition is an emerging necessity of in the industry. How- ever, traditional Computer Vision, which typically uses optical sensors, often fails in this task, due to unstable light conditions and possible blockages. Radar- based sensors, however, are able to detect an object in low-light conditions, as well as penetrate through blockages. Therefore, in this project, we focus on radar-based gesture recognition. We used TI’s AW1642 FMCW radar to collect data from three volunteers, per- forming 7 common gestures in an indoor laboratory setting. A preprocessing pipeline was built to generate Micro-Doppler heatmaps, which describes the mo- tion of different part of human body. These are fed into our carefully-designed deep neural networks to perform gesture classification. We are able to achieve an average accuracy of 82.13% for the 7-gesture classification. However, there remains an important need to generalize the solution, i.e., to achieve good performance on people not seen in the training set. In our ex- periments, we found the gaps in the same gestures of different people. We proposed to solve this using domain adversarial training. This allows us to ex- tract person/domain-independent features. When testing with unseen individuals, we are able to improve the accuracy by over 8% for 7-gesture classification. Keywords: radar, FMCW radar, Deep-learning, Micro-Doppler, Gesture Classifi- cation, Domain Adaptation