Fusing heterogeneous modalities for video and image re-ranking

Multimedia documents in popular image and video sharing websites such as Flickr and Youtube are heterogeneous documents with diverse ways of representations and rich user-supplied information. In this paper, we investigate how the agreement among heterogeneous modalities can be exploited to guide da...

Full description

Saved in:
Bibliographic Details
Main Authors: TAN, Hung-Khoon, NGO, Chong-wah
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2011
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6518
https://ink.library.smu.edu.sg/context/sis_research/article/7521/viewcontent/1991996.1992011.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7521
record_format dspace
spelling sg-smu-ink.sis_research-75212022-01-10T03:54:22Z Fusing heterogeneous modalities for video and image re-ranking TAN, Hung-Khoon NGO, Chong-wah Multimedia documents in popular image and video sharing websites such as Flickr and Youtube are heterogeneous documents with diverse ways of representations and rich user-supplied information. In this paper, we investigate how the agreement among heterogeneous modalities can be exploited to guide data fusion. The problem of fusion is cast as the simultaneous mining of agreement from different modalities and adaptation of fusion weights to construct a fused graph from these modalities. An iterative framework based on agreement-fusion optimization is thus proposed. We plug in two well-known algorithms: random walk and semi-supervised learning to this framework to illustrate the idea of how agreement (conflict) is incorporated (compromised) in the case of uniform and adaptive fusion. Experimental results on web video and image re-ranking demonstrate that, by proper fusion strategy rather than simple linear fusion, performance improvement on search can generally be expected. 2011-04-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6518 info:doi/10.1145/1991996.1992011 https://ink.library.smu.edu.sg/context/sis_research/article/7521/viewcontent/1991996.1992011.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University graph fusion heterogeneous modality fusion modality agreement re-ranking Data Storage Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic graph fusion
heterogeneous modality fusion
modality agreement
re-ranking
Data Storage Systems
Graphics and Human Computer Interfaces
spellingShingle graph fusion
heterogeneous modality fusion
modality agreement
re-ranking
Data Storage Systems
Graphics and Human Computer Interfaces
TAN, Hung-Khoon
NGO, Chong-wah
Fusing heterogeneous modalities for video and image re-ranking
description Multimedia documents in popular image and video sharing websites such as Flickr and Youtube are heterogeneous documents with diverse ways of representations and rich user-supplied information. In this paper, we investigate how the agreement among heterogeneous modalities can be exploited to guide data fusion. The problem of fusion is cast as the simultaneous mining of agreement from different modalities and adaptation of fusion weights to construct a fused graph from these modalities. An iterative framework based on agreement-fusion optimization is thus proposed. We plug in two well-known algorithms: random walk and semi-supervised learning to this framework to illustrate the idea of how agreement (conflict) is incorporated (compromised) in the case of uniform and adaptive fusion. Experimental results on web video and image re-ranking demonstrate that, by proper fusion strategy rather than simple linear fusion, performance improvement on search can generally be expected.
format text
author TAN, Hung-Khoon
NGO, Chong-wah
author_facet TAN, Hung-Khoon
NGO, Chong-wah
author_sort TAN, Hung-Khoon
title Fusing heterogeneous modalities for video and image re-ranking
title_short Fusing heterogeneous modalities for video and image re-ranking
title_full Fusing heterogeneous modalities for video and image re-ranking
title_fullStr Fusing heterogeneous modalities for video and image re-ranking
title_full_unstemmed Fusing heterogeneous modalities for video and image re-ranking
title_sort fusing heterogeneous modalities for video and image re-ranking
publisher Institutional Knowledge at Singapore Management University
publishDate 2011
url https://ink.library.smu.edu.sg/sis_research/6518
https://ink.library.smu.edu.sg/context/sis_research/article/7521/viewcontent/1991996.1992011.pdf
_version_ 1770575980676513792