3D content recovery and complexity reduction

• Main Author: Tan, Cheen Hau
• Other Authors: Chau Lap Pui
• Format: Theses and Dissertations
• Language: English
• Published: 2015
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Computer graphics; DRNTU::Engineering::Computer science and engineering::Computing methodologies::Pattern recognition
• Online Access: https://hdl.handle.net/10356/62230
• Institution: Nanyang Technological University

3D content, such as motion capture data and 3D meshes, are widely used in a number of sectors, such as entertainment and sports. Due to issues related to the acquisition of 3D content, postprocessing is often required on captured 3D content before it can be used in applications. For human motion capture (mocap) data acquired using optical systems, entries of the data might be missing due to occluded body parts or markers. For scanned 3D meshes, the complexity, or resolution, of the mesh could be much higher than necessary for the intended application; this will consume a large amount of computational resources. Hence, this thesis focuses on using mocap recovery algorithms to recover any missing mocap data, and on using mesh simplification algorithms to reduce the resolution of meshes. For mocap recovery, we focus on using low rank matrix completion with Singular Value Thresholding (SVT) optimization to recover mocap data. Previously, mocap data is arranged in the form of a matrix, where each frame forms a column to allow low rank matrix completion to recover missing entries. We present three strategies to extend and improve the performance of this mocap recovery framework, namely using a trajectory-based matrix representation, applying skeleton constraints, and using subspace constraints. Our mocap recovery methods target two types of missing data: random missing data, where each joint in a sequence are missing at random, and block missing data, where each joint is missing for long intervals of time. For the case of random missing data, we propose to arrange the mocap data matrix into columns of short trajectories, which we call the trajectory-based representation, for matrix completion recovery. We show that this representation produces a matrix with a lower rank than the previous frame-based matrix representation. Since matrix completion performs better on matrices with lower rank, this fact allows the SVT matrix completion method, by using the proposed representation, to recover missing mocap data at a much lower error. Both frame-based and trajectory-based matrix completion exploit different types of correlations; frame-based matrix completion relies on the correlation between different frames, whereas trajectory-based matrix completion relies on the correlation between trajectories. We propose to exploit both types of correlation simultaneously by constraining the solution of trajectory-based matrix completion in the subspace formed by the solution of frame-based matrix completion. The proposed method shows significant improvement over both frame-based and trajectory-based matrix completion. For block missing data, the effectiveness of matrix completion in recovering missing data decreases significantly when mocap data entries are missing for extended periods of time. To alleviate this problem, we exploit the fact that human bones are rigid and constrain inter-joint distances of connected joints. To this end, we extend the SVT matrix completion method to include skeleton constraints. The proposed method improves on the Singular Value Thresholding method significantly, especially when mocap data joints are missing for many consecutive frames. Image-Driven Simplification is a 3D mesh simplification method that simplifies meshes based on their visual appearance. It renders images of a mesh from each of 20 surrounding viewpoints to estimate the visual appearance of the mesh. Hence, this method is very time consuming since it requires repeated renders and image readback cycles during simplification. We propose to accelerate Image-Driven Simplification by using only one adaptively placed viewpoint instead of 20 viewpoints with fixed locations. Two computationally efficient methods are proposed to dynamically compute the viewpoint location so that simplification performance is not compromised. Both single viewpoint simplification methods run at an estimated twenty-five times faster than the original method at competitive simplification performance.