Body movement mimic – video-based human body motion transfer

In the age of rapid technological advancement, we have seen deepfake and even fallen for them. While many of the famous deepfake videos are for laughs and entertainment, such technology can significantly transform many areas of society today. In this report, we focus on the human body motion tran...

وصف كامل

محفوظ في:
التفاصيل البيبلوغرافية
المؤلف الرئيسي: Mamuduri, Paulani
مؤلفون آخرون: Cham Tat Jen
التنسيق: Final Year Project
اللغة:English
منشور في: Nanyang Technological University 2023
الموضوعات:
الوصول للمادة أونلاين:https://hdl.handle.net/10356/166084
الوسوم: إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
المؤسسة: Nanyang Technological University
اللغة: English
الوصف
الملخص:In the age of rapid technological advancement, we have seen deepfake and even fallen for them. While many of the famous deepfake videos are for laughs and entertainment, such technology can significantly transform many areas of society today. In this report, we focus on the human body motion transfer, whereby the motion of a body in a driving video is transferred onto a body in a 2D image. We have studied current technology, First Order Motion Model, which performs well for faces but not human body motion. We have proposed two approaches to improve model performance on the human body. The first approach used depth maps from pre-trained depth estimator MiDaS to guide keypoint detection in the First Order Motion Model. It did drastically help improve motion transfer, given that the quality of depth maps is good. The second approach trained the model with different keypoints to find an optimal one, given the nature of the dataset. Results show that the nature of the datasets affects the model performances.