Pose- and Attribute-consistent Person Image Synthesis

• Main Authors: XU, Cheng, CHEN, Zejun, MAI, Jiajie, XU, Xuemiao, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Computing methodologies; Artificial intelligence; Computer vision; Computer graphics; Image manipulation; Image processing; Artificial Intelligence and Robotics
• Online Access: https://ink.library.smu.edu.sg/sis_research/8369; https://ink.library.smu.edu.sg/context/sis_research/article/9372/viewcontent/Pose__and_Attribute_consistent_Person_Image_Synthesis.pdf
• Institution: Singapore Management University

PersonImageSynthesisaimsattransferringtheappearanceofthesourcepersonimageintoatargetpose. Existingmethods cannot handle largeposevariations and therefore suffer fromtwocritical problems: (1)synthesisdistortionduetotheentanglementofposeandappearanceinformationamongdifferentbody componentsand(2)failureinpreservingoriginalsemantics(e.g.,thesameoutfit).Inthisarticle,weexplicitly addressthesetwoproblemsbyproposingaPose-andAttribute-consistentPersonImageSynthesisNetwork (PAC-GAN).Toreduceposeandappearancematchingambiguity,weproposeacomponent-wisetransferring modelconsistingoftwostages.Theformerstagefocusesonlyonsynthesizingtargetposes,whilethelatter renderstargetappearancesbyexplicitlytransferringtheappearanceinformationfromthesourceimageto thetargetimageinacomponent-wisemanner. Inthisway,source-targetmatchingambiguityiseliminated duetothecomponent-wisedisentanglementofposeandappearancesynthesis.Second,tomaintainattribute consistency,werepresenttheinputimageasanattributevectorandimposeahigh-levelsemanticconstraint usingthisvectortoregularizethetargetsynthesis.ExtensiveexperimentalresultsontheDeepFashiondataset demonstratethesuperiorityofourmethodoverthestateoftheart,especiallyformaintainingposeandattributeconsistenciesunderlargeposevariations.