Body movement mimic – video-based human body motion transfer
In the age of rapid technological advancement, we have seen deepfake and even fallen for them. While many of the famous deepfake videos are for laughs and entertainment, such technology can significantly transform many areas of society today. In this report, we focus on the human body motion tran...
محفوظ في:
المؤلف الرئيسي: | |
---|---|
مؤلفون آخرون: | |
التنسيق: | Final Year Project |
اللغة: | English |
منشور في: |
Nanyang Technological University
2023
|
الموضوعات: | |
الوصول للمادة أونلاين: | https://hdl.handle.net/10356/166084 |
الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
المؤسسة: | Nanyang Technological University |
اللغة: | English |
الملخص: | In the age of rapid technological advancement, we have seen deepfake and even fallen for them. While
many of the famous deepfake videos are for laughs and entertainment, such technology can significantly
transform many areas of society today. In this report, we focus on the human body motion transfer,
whereby the motion of a body in a driving video is transferred onto a body in a 2D image. We have
studied current technology, First Order Motion Model, which performs well for faces but not human
body motion. We have proposed two approaches to improve model performance on the human body.
The first approach used depth maps from pre-trained depth estimator MiDaS to guide keypoint detection
in the First Order Motion Model. It did drastically help improve motion transfer, given that the quality
of depth maps is good. The second approach trained the model with different keypoints to find an
optimal one, given the nature of the dataset. Results show that the nature of the datasets affects the
model performances. |
---|