DESIGN AND IMPLEMENTATION OF POSE, FACE, AND HAND LANDMARK TRANSLATION LAYER FROM REAL-TIME VIDEO TO 3D AVATAR ON MOBILE DEVICE

• Main Author: Yudhistira Ramadhan, Enriko
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/79797
• Institution: Institut Teknologi Bandung

Online presence in the digital space used to be only in the form of profile pictures. Advancement in the digital age pushes the possibility of customizing internet users’ online presence to be more customizable. This representation of a person is called an “Avatar”. An avatar usually comes in 2D and 3D form. Using avatar, an individual can change their appearance as to their liking. The next step is to make an avatar interactive, as in following the user’s movement. The technology to make an interactive 3D avatar is not new, but it needs a separate sensor and an additional motion capture suit. To make 3D avatar a ubiquitous technology and adopted by the masses, it needs to be easier to set up and use. The sensors and other motion capture device needs to be eliminated and the technology needs to be usable with only a single, ordinary camera (preferrably a mobile phone camera). The task is not as simple as eliminating additional sensors. The task of capturing the movement of a subject needs to be substituted with something else. Image processing technology can be used to predict movement of a subject in a video stream, removing the needs for additional sensor. After image processing is used, captured data points needs to be associated with 3D avatar. The 3D avatar then would move accordingly with the data points. This whole process would have to be lightweight and fast enough to run on a mobile phone. The testing result shows that while the proposed system has errors in some scenario, it is adequate for casual use. The system performs poorly where body parts’ occlusions happen.