Multi-view face synthesis via progressive face flow

Existing GAN-based multi-view face synthesis methods rely heavily on "creating" faces, and thus they struggle in reproducing the faithful facial texture and fail to preserve identity when undergoing a large angle rotation. In this paper, we combat this problem by dividing the challenging l...

Full description

Saved in:
Bibliographic Details
Main Authors: XU, Yangyang, XU, Xuemiao, JIAO, Jianbo, LI, Keke, XU, Cheng, HE, Shengfeng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2021
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7875
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8878
record_format dspace
spelling sg-smu-ink.sis_research-88782023-06-15T09:00:05Z Multi-view face synthesis via progressive face flow XU, Yangyang XU, Xuemiao JIAO, Jianbo LI, Keke XU, Cheng HE, Shengfeng Existing GAN-based multi-view face synthesis methods rely heavily on "creating" faces, and thus they struggle in reproducing the faithful facial texture and fail to preserve identity when undergoing a large angle rotation. In this paper, we combat this problem by dividing the challenging large-angle face synthesis into a series of easy small-angle rotations, and each of them is guided by a face flow to maintain faithful facial details. In particular, we propose a Face Flow-guided Generative Adversarial Network (FFlowGAN) that is specifically trained for small-angle synthesis. The proposed network consists of two modules, a face flow module that aims to compute a dense correspondence between the input and target faces. It provides strong guidance to the second module, face synthesis module, for emphasizing salient facial texture. We apply FFlowGAN multiple times to progressively synthesize different views, and therefore facial features can be propagated to the target view from the very beginning. All these multiple executions are cascaded and trained end-to-end with a unified back-propagation, and thus we ensure each intermediate step contributes to the final result. Extensive experiments demonstrate the proposed divide-and-conquer strategy is effective, and our method outperforms the state-of-the-art on four benchmark datasets qualitatively and quantitatively. 2021-01-01T08:00:00Z text https://ink.library.smu.edu.sg/sis_research/7875 info:doi/10.1109/TIP.2021.3090658 Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Faces;Face recognition;Generative adversarial networks;Image reconstruction;Facial features;Deep learning;Three-dimensional displays;Multi-view face synthesis;pose-invariant face recognition;face reconstruction Information Security
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Faces;Face recognition;Generative adversarial networks;Image reconstruction;Facial features;Deep learning;Three-dimensional displays;Multi-view face synthesis;pose-invariant face recognition;face reconstruction
Information Security
spellingShingle Faces;Face recognition;Generative adversarial networks;Image reconstruction;Facial features;Deep learning;Three-dimensional displays;Multi-view face synthesis;pose-invariant face recognition;face reconstruction
Information Security
XU, Yangyang
XU, Xuemiao
JIAO, Jianbo
LI, Keke
XU, Cheng
HE, Shengfeng
Multi-view face synthesis via progressive face flow
description Existing GAN-based multi-view face synthesis methods rely heavily on "creating" faces, and thus they struggle in reproducing the faithful facial texture and fail to preserve identity when undergoing a large angle rotation. In this paper, we combat this problem by dividing the challenging large-angle face synthesis into a series of easy small-angle rotations, and each of them is guided by a face flow to maintain faithful facial details. In particular, we propose a Face Flow-guided Generative Adversarial Network (FFlowGAN) that is specifically trained for small-angle synthesis. The proposed network consists of two modules, a face flow module that aims to compute a dense correspondence between the input and target faces. It provides strong guidance to the second module, face synthesis module, for emphasizing salient facial texture. We apply FFlowGAN multiple times to progressively synthesize different views, and therefore facial features can be propagated to the target view from the very beginning. All these multiple executions are cascaded and trained end-to-end with a unified back-propagation, and thus we ensure each intermediate step contributes to the final result. Extensive experiments demonstrate the proposed divide-and-conquer strategy is effective, and our method outperforms the state-of-the-art on four benchmark datasets qualitatively and quantitatively.
format text
author XU, Yangyang
XU, Xuemiao
JIAO, Jianbo
LI, Keke
XU, Cheng
HE, Shengfeng
author_facet XU, Yangyang
XU, Xuemiao
JIAO, Jianbo
LI, Keke
XU, Cheng
HE, Shengfeng
author_sort XU, Yangyang
title Multi-view face synthesis via progressive face flow
title_short Multi-view face synthesis via progressive face flow
title_full Multi-view face synthesis via progressive face flow
title_fullStr Multi-view face synthesis via progressive face flow
title_full_unstemmed Multi-view face synthesis via progressive face flow
title_sort multi-view face synthesis via progressive face flow
publisher Institutional Knowledge at Singapore Management University
publishDate 2021
url https://ink.library.smu.edu.sg/sis_research/7875
_version_ 1770576574018486272