Click-through-based cross-view learning for image search

One of the fundamental problems in image search is to rank image documents according to a given textual query. Existing search engines highly depend on surrounding texts for ranking images, or leverage the query-image pairs annotated by human labelers to train a series of ranking functions. However,...

Full description

Saved in:
Bibliographic Details
Main Authors: PAN, Yingwei, YAO, Ting, MEI, Tao, LI, Houqiang, NGO, Chong-wah, RUI, Yong
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2014
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6514
https://ink.library.smu.edu.sg/context/sis_research/article/7517/viewcontent/2600428.2609568.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7517
record_format dspace
spelling sg-smu-ink.sis_research-75172022-01-10T03:56:07Z Click-through-based cross-view learning for image search PAN, Yingwei YAO, Ting MEI, Tao LI, Houqiang NGO, Chong-wah RUI, Yong One of the fundamental problems in image search is to rank image documents according to a given textual query. Existing search engines highly depend on surrounding texts for ranking images, or leverage the query-image pairs annotated by human labelers to train a series of ranking functions. However, there are two major limitations: 1) the surrounding texts are often noisy or too few to accurately describe the image content, and 2) the human annotations are resourcefully expensive and thus cannot be scaled up. We demonstrate in this paper that the above two fundamental challenges can be mitigated by jointly exploring the cross-view learning and the use of click-through data. The former aims to create a latent subspace with the ability in comparing information from the original incomparable views (i.e., textual and visual views), while the latter explores the largely available and freely accessible click-through data (i.e., “crowdsourced” human intelligence) for understanding query. Specifically, we propose a novel cross-view learning method for image search, named Click-through-based Crossview Learning (CCL), by jointly minimizing the distance between the mappings of query and image in the latent subspace and preserving the inherent structure in each original space. On a large-scale click-based image dataset, CCL achieves the improvement over Support Vector Machinebased method by 4.0% in terms of relevance, while reducing the feature dimension by several orders of magnitude (e.g., from thousands to tens). Moreover, the experiments also demonstrate the superior performance of CCL to several state-of-the-art subspace learning techniques. 2014-07-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6514 info:doi/10.1145/2600428.2609568 https://ink.library.smu.edu.sg/context/sis_research/article/7517/viewcontent/2600428.2609568.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Clickthrough data Cross-view learning DNN image representation Image search Subspace learning Data Storage Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Clickthrough data
Cross-view learning
DNN image representation
Image search
Subspace learning
Data Storage Systems
Graphics and Human Computer Interfaces
spellingShingle Clickthrough data
Cross-view learning
DNN image representation
Image search
Subspace learning
Data Storage Systems
Graphics and Human Computer Interfaces
PAN, Yingwei
YAO, Ting
MEI, Tao
LI, Houqiang
NGO, Chong-wah
RUI, Yong
Click-through-based cross-view learning for image search
description One of the fundamental problems in image search is to rank image documents according to a given textual query. Existing search engines highly depend on surrounding texts for ranking images, or leverage the query-image pairs annotated by human labelers to train a series of ranking functions. However, there are two major limitations: 1) the surrounding texts are often noisy or too few to accurately describe the image content, and 2) the human annotations are resourcefully expensive and thus cannot be scaled up. We demonstrate in this paper that the above two fundamental challenges can be mitigated by jointly exploring the cross-view learning and the use of click-through data. The former aims to create a latent subspace with the ability in comparing information from the original incomparable views (i.e., textual and visual views), while the latter explores the largely available and freely accessible click-through data (i.e., “crowdsourced” human intelligence) for understanding query. Specifically, we propose a novel cross-view learning method for image search, named Click-through-based Crossview Learning (CCL), by jointly minimizing the distance between the mappings of query and image in the latent subspace and preserving the inherent structure in each original space. On a large-scale click-based image dataset, CCL achieves the improvement over Support Vector Machinebased method by 4.0% in terms of relevance, while reducing the feature dimension by several orders of magnitude (e.g., from thousands to tens). Moreover, the experiments also demonstrate the superior performance of CCL to several state-of-the-art subspace learning techniques.
format text
author PAN, Yingwei
YAO, Ting
MEI, Tao
LI, Houqiang
NGO, Chong-wah
RUI, Yong
author_facet PAN, Yingwei
YAO, Ting
MEI, Tao
LI, Houqiang
NGO, Chong-wah
RUI, Yong
author_sort PAN, Yingwei
title Click-through-based cross-view learning for image search
title_short Click-through-based cross-view learning for image search
title_full Click-through-based cross-view learning for image search
title_fullStr Click-through-based cross-view learning for image search
title_full_unstemmed Click-through-based cross-view learning for image search
title_sort click-through-based cross-view learning for image search
publisher Institutional Knowledge at Singapore Management University
publishDate 2014
url https://ink.library.smu.edu.sg/sis_research/6514
https://ink.library.smu.edu.sg/context/sis_research/article/7517/viewcontent/2600428.2609568.pdf
_version_ 1770575979943559168