3D human mesh recovery system for motor function assessment

• Main Author: Wang, Ruisi
• Other Authors: Liu Ziwei
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science
• Online Access: https://hdl.handle.net/10356/175963
• Institution: Nanyang Technological University

Stroke is a leading cause of disabilities worldwide, with hemiplegia emerging as the prevalent impairment after stroke. Such impairment is associated with restricted activities and worse health-related quality of life. Traditional rehabilitation assessment methods are qualitative, such as Fugl-Meyer scale. Generally, different therapists have different evaluations. Additionally, it is labour-intensive and time-consuming, including a large number of repetitive body movements with the assistance of doctors or professional therapists. This report proposes an automated upper extremity motor function assessment system that can be practically used in a clinical environment, utilizing three calibrated Azure Kinect cameras. Benefits from SMPLer-X, a generalist foundation model for monocular motion capture, patients can be reconstructed in 3D space. Based on the confidence level of 3D joints, the resultant parametric human model (SMPL-X) undergoes further optimization using multi-view images, offering a detailed representation of the patient’s body pose and joint positions. Following the Fugl-Meyer Assessment (FMA) guidelines, a rule-based logic classification algorithm has been developed to automatically assign FMA scores using the extracted features obtained from the Kinect cameras. Furthermore, we have adapted this pose estimation system for mobile phones to enhance its accessibility and usability for future development. Normal assessment such as Upper limb rehabilitation requires long-term, repetitive rehabilitation training and assessment. Most stroke survivors cannot receive adequate outpatient stroke rehabilitation due to barriers including costs, travel and limited use of public transportation. The future work aims to develop a home-based virtual rehabilitation system that could be a useful alternative for conventional rehabilitation to overcome barriers for outpatient rehabilitation.