Zero-shot learning via category-specific visual-semantic mapping and label refinement

• Main Authors: Niu, Li, Cai, Jianfei, Veeraraghavan, Ashok, Zhang, Liqing
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2020
• Subjects: Engineering::Computer science and engineering; Zero-shot Learning (ZSL); Domain Adaptation
• Online Access: https://hdl.handle.net/10356/142785
• Institution: Nanyang Technological University

Zero-shot learning (ZSL) aims to classify a test instance from an unseen category based on the training instances from seen categories in which the gap between seen categories and unseen categories is generally bridged via visual-semantic mapping between the low-level visual feature space and the intermediate semantic space. However, the visual-semantic mapping (i.e., projection) learnt based on seen categories may not generalize well to unseen categories, which is known as the projection domain shift in ZSL. To address this projection domain shift issue, we propose a method named adaptive embedding ZSL (AEZSL) to learn an adaptive visual-semantic mapping for each unseen category, followed by progressive label refinement. Moreover, to avoid learning visual-semantic mapping for each unseen category in the large-scale classification task, we additionally propose a deep adaptive embedding model named deep AEZSL sharing the similar idea (i.e., visual-semantic mapping should be category specific and related to the semantic space) with AEZSL, which only needs to be trained once, but can be applied to arbitrary number of unseen categories. Extensive experiments demonstrate that our proposed methods achieve the state-of-the-art results for image classification on three small-scale benchmark datasets and one large-scale benchmark dataset.