MOTION CAPTURE SYSTEM ON DIGITAL CREATIVE MEDIA USING RGB CAMERAS

• Main Author: Ozer, Chokyi
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/66602
• Institution: Institut Teknologi Bandung

Full body motion capture is an animation creation method that is currently inaccessible by most content creators. This problem is resulted from the high cost and hard to use equipment needed to produce these types of animations. On the other hand, facial motion capture technology has really been advancing, so much so that it is even used by non-animators. This final project aims to combine the simplicity and ease-of-use of facial motion capture systems with the capability of creating a full body animation. All this should be able to be achieved with only one to two RGB cameras in the animation process. To achieve this, the end-product will make use of machine learning based pose estimation technology such as the ones provided by MediaPipe. The inputs from the RGB cameras will be configured according to prior calibration done to each camera, after which the skeleton-based 3D pose will be extracted using that machine learning model. Then, each joint on the avatar will be rotated inside Unity game engine according to the position of the obtained keypoints. From the testing results, we obtained that the immediate 3D pose estimation method gives the most optimal results from the various methods that have been experimented on. Even so, the system still lacks in terms of speed of creating animation frames and still has difficulties on detecting the depth of the keypoint locations.