High-resolution face swapping via latent semantics disentanglement
We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitl...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2022
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/8532 https://ink.library.smu.edu.sg/context/sis_research/article/9535/viewcontent/Xu_High_Resolution_Face_Swapping_via_Latent_Semantics_Disentanglement_CVPR_2022_paper.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-9535 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-95352024-01-22T14:58:20Z High-resolution face swapping via latent semantics disentanglement XU, Yangyang DENG, Bailin WANG, Junle JING, Yanqing PAN, Jia HE, Shengfeng We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending to produce a plausible swapping result. We further extend our method to video face swapping by enforcing two spatio-temporal constraints on the latent space and the image space. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art image/video face swapping methods in terms of hallucination quality and consistency. Code can be found at: https://github.com/cnnlstm/FSLSD_HiRes. 2022-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8532 info:doi/10.1109/CVPR52688.2022.00749 https://ink.library.smu.edu.sg/context/sis_research/article/9535/viewcontent/Xu_High_Resolution_Face_Swapping_via_Latent_Semantics_Disentanglement_CVPR_2022_paper.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Image and video synthesis and generation Face and gestures Low-level vision Computer vision Codes Face recognition Semantics Generators Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Image and video synthesis and generation Face and gestures Low-level vision Computer vision Codes Face recognition Semantics Generators Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
spellingShingle |
Image and video synthesis and generation Face and gestures Low-level vision Computer vision Codes Face recognition Semantics Generators Artificial Intelligence and Robotics Graphics and Human Computer Interfaces XU, Yangyang DENG, Bailin WANG, Junle JING, Yanqing PAN, Jia HE, Shengfeng High-resolution face swapping via latent semantics disentanglement |
description |
We present a novel high-resolution face swapping method using the inherent prior knowledge of a pre-trained GAN model. Although previous research can leverage generative priors to produce high-resolution results, their quality can suffer from the entangled semantics of the latent space. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator, deriving structure at-tributes from the shallow layers and appearance attributes from the deeper ones. Identity and pose information within the structure attributes are further separated by introducing a landmark-driven structure transfer latent direction. The disentangled latent code produces rich generative features that incorporate feature blending to produce a plausible swapping result. We further extend our method to video face swapping by enforcing two spatio-temporal constraints on the latent space and the image space. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art image/video face swapping methods in terms of hallucination quality and consistency. Code can be found at: https://github.com/cnnlstm/FSLSD_HiRes. |
format |
text |
author |
XU, Yangyang DENG, Bailin WANG, Junle JING, Yanqing PAN, Jia HE, Shengfeng |
author_facet |
XU, Yangyang DENG, Bailin WANG, Junle JING, Yanqing PAN, Jia HE, Shengfeng |
author_sort |
XU, Yangyang |
title |
High-resolution face swapping via latent semantics disentanglement |
title_short |
High-resolution face swapping via latent semantics disentanglement |
title_full |
High-resolution face swapping via latent semantics disentanglement |
title_fullStr |
High-resolution face swapping via latent semantics disentanglement |
title_full_unstemmed |
High-resolution face swapping via latent semantics disentanglement |
title_sort |
high-resolution face swapping via latent semantics disentanglement |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2022 |
url |
https://ink.library.smu.edu.sg/sis_research/8532 https://ink.library.smu.edu.sg/context/sis_research/article/9535/viewcontent/Xu_High_Resolution_Face_Swapping_via_Latent_Semantics_Disentanglement_CVPR_2022_paper.pdf |
_version_ |
1789483260148973568 |