Markerless motion capture and analysis based on depth images

• Main Author: Bian, Zhenpeng
• Other Authors: Nadia Magnenat-Thalmann
• Format: Theses and Dissertations
• Language: English
• Published: 2016
• Subjects: DRNTU::Engineering::Electrical and electronic engineering::Electronic systems::Signal processing; DRNTU::Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; DRNTU::Engineering::Computer science and engineering::Information systems::Information systems applications; DRNTU::Engineering::Electrical and electronic engineering::Applications of electronics
• Online Access: https://hdl.handle.net/10356/65865
• Institution: Nanyang Technological University

The works presented in this thesis focus on depth images based human motion capture in realistic daily scenarios and two novel motion analysis frameworks on fall detection and human-computer interface based on motion capture algorithms. We propose a body part extraction algorithm to improve the robustness, speed and accuracy for human motion capture compared to existing methods. Based on the new motion capture method, we propose two motion analysis frameworks, resulting in better performance than the state-of-the-art methods, especially their robustness and low computational complexity. The major contributions of this thesis are: (i) A fast training algorithm based on a compact feature for markerless motion capture. (ii) A robust fall detection framework based on markerless motion capture. (iii) A convenient and robust human computer interface for people with disabled hands. Traditional motion capture approaches obtain the information from the motion of markers, which are mostly used in laboratory environments by professional users. More and more applications require the motion capture techniques, especially the markerless motion capture techniques. This thesis develops markerless motion capture methods using depth images, which can produce a low cost and convenient system for motion capture applications. To capture the motion of the human body, the human body parts are extracted by the proposed Randomized Decision Tree (RDT) algorithm. The training phase affects the robustness and accuracy of the RDT algorithm. The dataset for training of the RDT algorithm should be large enough to cover enough cases. One of the advantages of the depth image framework is that it is easier to build up large realistic training dataset compared with the RGB image framework. Furthermore, the depth images are independent of colour and lighting illumination. However, one of the challenges of the RDT algorithms is the high computational complexity of training, which is caused by the huge amount of the feature candidates for each training pixel. We propose to reduce the amount of the feature candidates by search algorithms based on a compact feature. This fast training algorithm can dramatically speed up the training phase without loss of accuracy in the test phase. The first novel motion analysis framework based on motion capture is fall detection. Researches have shown that getting help quickly after a fall can significantly reduce the hospitalization risk and the death risk for elderly people. Based on the markerless motion capture technology, we propose a real-time fall detection framework by analysing the 3D head trajectory. This system is more convenient since it is a non-intrusive approach compared to the wearable sensor based fall detection approaches. There are two major challenges of the fall detection: the daily activities are very variable, and the environmental illumination is complexly. The proposed body part extraction algorithm can track the human body in different human poses, and the proposed system is independent of illumination. Then, the 3D head trajectory is analysed by the Support Vector Machine (SVM) classifier. The second framework is the human computer interface. The people with disabled hands are very difficult to use personal computer's standard interfaces. Therefore, we propose a human computer interface for the persons with tetraplegia based on markerless motion capture via a single depth camera. The nose position navigates the cursor of computer, and the mouth status triggers the input commands. Compared with many other Assistive Technologies (ATs) for persons with tetraplegia, this system does not need calibration, and enables users to adjust their head postures relative to the cursor freely. The proposed method can detect the nose position and the mouth status in a single depth image, and can avoid the problem of feature drift, which exists in most tracking approaches. During the training phase, the mouth is labeled according to the status of the mouth and the nose is set as a single label. During the test phase, the nose position and the mouth status can be extracted after classifying each pixel. Compared to the landmark-based mouth status detection, the proposed mouth status detection is more efficient and robust. The experimental results show that the performance of the proposed interface is superior to the stat-of-the-art ATs.