GANmut: learning interpretable conditional space for a gamut of emotions

• Main Authors: D'APOLITO, S., PAUNDEL, D.P., HUANG, Zhiwu, VERGARA, A.R., VAN, Gool L.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/6409; https://ink.library.smu.edu.sg/context/sis_research/article/7412/viewcontent/GANmutLearning_Interpretable_Conditional_Space_for_Gamut_of_Emotions.pdf
• Institution: Singapore Management University

Humans can communicate emotions through a plethora of facial expressions, each with its own intensity, nuances and ambiguities. The generation of such variety by means of conditional GANs is limited to the expressions encoded in the used label system. These limitations are caused either due to burdensome labeling demand or the confounded label space. On the other hand, learning from inexpensive and intuitive basic categorical emotion labels leads to limited emotion variability. In this paper, we propose a novel GAN-based framework which learns an expressive and interpretable conditional space (usable as a label space) of emotions, instead of conditioning on handcrafted labels. Our framework only uses the categorical labels of basic emotions to jointly learn the conditional space as well as the emotion manipulation. Such learning can benefit from the image variability within discrete labels, especially when the intrinsic labels reside beyond the discrete space of the defined. Our experiments demonstrate the effectiveness of the proposed framework, by allowing us to control and generate a gamut of complex and compound emotions, while using only the basic categorical emotion labels during training.