Personalized markerless upper-body tracking with a depth camera and wrist-worn inertial measurement units

• Main Authors: Jatesiktat, Prayook, Anopas, Dollaporn, Ang, Wei Tech
• Other Authors: School of Mechanical and Aerospace Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2020
• Subjects: Engineering::Mechanical engineering; Markerless Motion Capture Technique; Inertial Measurement Units
• Online Access: https://hdl.handle.net/10356/137350
• Institution: Nanyang Technological University

A markerless motion capture technique is proposed based on a fusion between a depth camera (Kinect V2) and a pair of wrist-worn inertial measurement units (IMU). The method creates a personalized articulated human mesh model from one depth image frame and uses that model to improve the accuracy of the upper-body joint tracking. The IMUs are useful as an additional clue for the arm tracking, especially during an occlusion. An evaluation of the method against a marker-based system as a gold standard using data from 6 subjects is done. The result shows over 20% reduction in upper-limb joint position errors when compared to Kinect's skeleton tracking. All the collected data are calibrated, synchronized, and made publicly available for research purposes.