Supervised generative adversarial cross-modal hashing by transferring pairwise similarities for venue discovery

• Main Authors: AGGARWAL, Himanshu, SHAH, Rajiv Ratn, TANG, Suhua, ZHU, Feida
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/4840
• Institution: Singapore Management University

Venue discovery using real-world multimedia data has not been investigated thoroughly. We are referring to business and travel locations as venues in this study and aim to improve the efficiency of venue discovery by hashing. Most existing supervised cross-modal hashing methods map data in different modalities to Hamming space, where the semantic information is exploited to supervise data of different modalities during the training stage. However, previous works neglect pairwise similarity between data in different modalities, which lead to degraded performance of hashing function learning. To address this issue, we propose a supervised Generative Adversarial Cross-modal Hashing method by Transferring Pairwise Similarities (SGACH-TPS). This work has three significant contributions: i) we propose a model for making efficient venue discovery, ii) the supervised generative adversarial network can construct a hash function to map multimodal data to a common hamming space. iii) a simple transfer training strategy for the adversarial network is suggested to supervise data in different modalities where the pairwise similarity is transferred to the fine-tuning stage of training. Evaluation on the new WikiVenue dataset confirms the superiority of the proposed method.