Holistically associated transductive zero-shot learning

• Main Authors: XU, Yangyang, XU, Xuemiao, HAN, Guoqiang, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Visualization; Semantics; Artificial neural networks; Predictive models; Training; Pairwise error probability; Loss measurement; Affinity matrix; class association; instance association; zero-shot learning (ZSL); Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/7862
• Institution: Singapore Management University

With the explosive growth of visual media categories, zero-shot learning (ZSL) aims to transfer the knowledge obtained from the seen classes to the unseen classes for recognizing novel instances. However, there is a domain gap between the seen and the unseen classes, and simply matching the unseen instances using nearest neighbor searching in the embedding space cannot bridge this gap effectively. In this article, we propose a holistically associated model to overcome this obstacle. In particular, the proposed model is designed to combat two fundamental problems of ZSL: 1) the representation learning and 2) label assignment of the unseen classes. The first problem is addressed by proposing an affinity propagation network, which considers holistic pairwise connections of all classes for producing exemplar features of the unseen samples. We cope with the second issue by proposing a progressive clustering module. It iteratively refines unseen clusters so that holistic unseen instance features can be used for a reliable classwise label assignment. Thanks to the precise exemplar features and classwise label assignment, our model eliminates the domain gap effectively. We extensively evaluate the proposed model on five human action and image data sets, i.e., Olympics Sports, HMDB51, UCF101, Animals with Attributes 2, and SUN. The experimental results show that the proposed model outperforms state-of-the-art methods on these substantially different data sets.