Appearance-preserved portrait-to-anime translation via proxy-guided domain adaptation

• Main Authors: XIAO, Wenpeng, XU, Cheng, MAI, Jiajie, XU, Xuemiao, LI, Yue, LI, Chengze, LIU, Xueting, Shengfeng HE
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Adaptation models; Coser portrait proxy; Deformable models; Direct mapping; Domain adaptation; Face; Portrait-to-anime translation; Shape; Simple++; Two domains; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/8359; https://ink.library.smu.edu.sg/context/sis_research/article/9362/viewcontent/Appearance_preserved_Portrait_to_anime_Translation_via_Proxy_guided_Domain_Adaptation.pdf
• Institution: Singapore Management University

Converting a human portrait to anime style is a desirable but challenging problem. Existing methods fail to resolve this problem due to the large inherent gap between two domains that cannot be overcome by a simple direct mapping. For this reason, these methods struggle to preserve the appearance features in the original photo. In this paper, we discover an intermediate domain, the coser portrait (portraits of humans costuming as anime characters), that helps bridge this gap. It alleviates the learning ambiguity and loosens the mapping difficulty in a progressive manner. Specifically, we start from learning the mapping between coser and anime portraits, and present a proxy-guided domain adaptation learning scheme with three progressive adaptation stages to shift the initial model to the human portrait domain. In this way, our model can generate visually pleasant anime portraits with well-preserved appearances given the human portrait. Our model adopts a disentangled design by breaking down the translation problem into two specific subtasks of face deformation and portrait stylization. This further elevates the generation quality. Extensive experimental results show that our model can achieve visually compelling translation with better appearance preservation and perform favorably against the existing methods both qualitatively and quantitatively.