Content-Based Visual Landmark Search via Multimodal Hypergraph Learning

While content-based landmark image search has recently received a lot of attention and became a very active domain, it still remains a challenging problem. Among the various reasons, high diverse visual content is the most significant one. It is common that for the same landmark, images with a wide...

Full description

Saved in:
Bibliographic Details
Main Authors: ZHU, Lei, SHEN, Jialie, JIN, Hai, ZHENG, Ran, XIE, Liang
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2015
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/2465
https://ink.library.smu.edu.sg/context/sis_research/article/3464/viewcontent/Content_basedVisualLandmarkSearch_2015.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-3464
record_format dspace
spelling sg-smu-ink.sis_research-34642017-03-22T09:35:08Z Content-Based Visual Landmark Search via Multimodal Hypergraph Learning ZHU, Lei SHEN, Jialie JIN, Hai ZHENG, Ran XIE, Liang While content-based landmark image search has recently received a lot of attention and became a very active domain, it still remains a challenging problem. Among the various reasons, high diverse visual content is the most significant one. It is common that for the same landmark, images with a wide range of visual appearances can be found from different sources and different landmarks may share very similar sets of images. As a consequence, it is very hard to accurately estimate the similarities between the landmarks purely based on single type of visual feature. Moreover, the relationships between landmark images can be very complex and how to develop an effective modeling scheme to characterize the associations still remains an open question. Motivated by these concerns, we propose multimodal hypergraph (MMHG) to characterize the complex associations between landmark images. In MMHG, images are modeled as independent vertices and hyperedges contain several vertices corresponding to particular views. Multiple hypergraphs are firstly constructed independently based on different visual modalities to describe the hidden high-order relations from different aspects. Then, they are integrated together to involve discriminative information from heterogeneous sources. We also propose a novel content-based visual landmark search system based on MMHG to facilitate effective search. Distinguished from the existing approaches, we design a unified computational module to support query-specific combination weight learning. An extensive experiment study on a large-scale test collection demonstrates the effectiveness of our scheme over state-of-the-art approaches. 2015-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/2465 info:doi/10.1109/TCYB.2014.2383389 https://ink.library.smu.edu.sg/context/sis_research/article/3464/viewcontent/Content_basedVisualLandmarkSearch_2015.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Content-based visual landmark search high-order relations multimodal hypergraph (MMHG) visual diversity Computer Sciences Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Content-based visual landmark search
high-order relations
multimodal hypergraph (MMHG)
visual diversity
Computer Sciences
Databases and Information Systems
spellingShingle Content-based visual landmark search
high-order relations
multimodal hypergraph (MMHG)
visual diversity
Computer Sciences
Databases and Information Systems
ZHU, Lei
SHEN, Jialie
JIN, Hai
ZHENG, Ran
XIE, Liang
Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
description While content-based landmark image search has recently received a lot of attention and became a very active domain, it still remains a challenging problem. Among the various reasons, high diverse visual content is the most significant one. It is common that for the same landmark, images with a wide range of visual appearances can be found from different sources and different landmarks may share very similar sets of images. As a consequence, it is very hard to accurately estimate the similarities between the landmarks purely based on single type of visual feature. Moreover, the relationships between landmark images can be very complex and how to develop an effective modeling scheme to characterize the associations still remains an open question. Motivated by these concerns, we propose multimodal hypergraph (MMHG) to characterize the complex associations between landmark images. In MMHG, images are modeled as independent vertices and hyperedges contain several vertices corresponding to particular views. Multiple hypergraphs are firstly constructed independently based on different visual modalities to describe the hidden high-order relations from different aspects. Then, they are integrated together to involve discriminative information from heterogeneous sources. We also propose a novel content-based visual landmark search system based on MMHG to facilitate effective search. Distinguished from the existing approaches, we design a unified computational module to support query-specific combination weight learning. An extensive experiment study on a large-scale test collection demonstrates the effectiveness of our scheme over state-of-the-art approaches.
format text
author ZHU, Lei
SHEN, Jialie
JIN, Hai
ZHENG, Ran
XIE, Liang
author_facet ZHU, Lei
SHEN, Jialie
JIN, Hai
ZHENG, Ran
XIE, Liang
author_sort ZHU, Lei
title Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
title_short Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
title_full Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
title_fullStr Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
title_full_unstemmed Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
title_sort content-based visual landmark search via multimodal hypergraph learning
publisher Institutional Knowledge at Singapore Management University
publishDate 2015
url https://ink.library.smu.edu.sg/sis_research/2465
https://ink.library.smu.edu.sg/context/sis_research/article/3464/viewcontent/Content_basedVisualLandmarkSearch_2015.pdf
_version_ 1770572184496898048