Near-duplicate keyframe retrieval with visual keywords and semantic context

Near-duplicate keyframes (NDK) play a unique role in large-scale video search, news topic detection and tracking. In this paper, we propose a novel NDK retrieval approach by exploring both visual and textual cues from the visual vocabulary and semantic context respectively. The vocabulary, which pro...

Full description

Saved in:
Bibliographic Details
Main Authors: WU, Xiao, ZHAO, Wan-Lei, NGO, Chong-wah
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2007
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6445
https://ink.library.smu.edu.sg/context/sis_research/article/7448/viewcontent/1282280.1282309.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7448
record_format dspace
spelling sg-smu-ink.sis_research-74482022-01-10T06:17:33Z Near-duplicate keyframe retrieval with visual keywords and semantic context WU, Xiao ZHAO, Wan-Lei NGO, Chong-wah Near-duplicate keyframes (NDK) play a unique role in large-scale video search, news topic detection and tracking. In this paper, we propose a novel NDK retrieval approach by exploring both visual and textual cues from the visual vocabulary and semantic context respectively. The vocabulary, which provides entries for visual keywords, is formed by the clustering of local keypoints. The semantic context is inferred from the speech transcript surrounding a keyframe. We experiment the usefulness of visual keywords and semantic context, separately and jointly, using cosine similarity and language models. By linearly fusing both modalities, performance improvement is reported compared with the techniques with keypoint matching. While matching suffers from expensive computation due to the need of online nearest neighbor search, our approach is effective and efficient enough for online video search. 2007-07-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6445 info:doi/10.1145/1282280.1282309 https://ink.library.smu.edu.sg/context/sis_research/article/7448/viewcontent/1282280.1282309.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Image retrieval Language model Multiple modalities Near-duplicate keyframe News videos Similarity measure Databases and Information Systems Theory and Algorithms
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Image retrieval
Language model
Multiple modalities
Near-duplicate keyframe
News videos
Similarity measure
Databases and Information Systems
Theory and Algorithms
spellingShingle Image retrieval
Language model
Multiple modalities
Near-duplicate keyframe
News videos
Similarity measure
Databases and Information Systems
Theory and Algorithms
WU, Xiao
ZHAO, Wan-Lei
NGO, Chong-wah
Near-duplicate keyframe retrieval with visual keywords and semantic context
description Near-duplicate keyframes (NDK) play a unique role in large-scale video search, news topic detection and tracking. In this paper, we propose a novel NDK retrieval approach by exploring both visual and textual cues from the visual vocabulary and semantic context respectively. The vocabulary, which provides entries for visual keywords, is formed by the clustering of local keypoints. The semantic context is inferred from the speech transcript surrounding a keyframe. We experiment the usefulness of visual keywords and semantic context, separately and jointly, using cosine similarity and language models. By linearly fusing both modalities, performance improvement is reported compared with the techniques with keypoint matching. While matching suffers from expensive computation due to the need of online nearest neighbor search, our approach is effective and efficient enough for online video search.
format text
author WU, Xiao
ZHAO, Wan-Lei
NGO, Chong-wah
author_facet WU, Xiao
ZHAO, Wan-Lei
NGO, Chong-wah
author_sort WU, Xiao
title Near-duplicate keyframe retrieval with visual keywords and semantic context
title_short Near-duplicate keyframe retrieval with visual keywords and semantic context
title_full Near-duplicate keyframe retrieval with visual keywords and semantic context
title_fullStr Near-duplicate keyframe retrieval with visual keywords and semantic context
title_full_unstemmed Near-duplicate keyframe retrieval with visual keywords and semantic context
title_sort near-duplicate keyframe retrieval with visual keywords and semantic context
publisher Institutional Knowledge at Singapore Management University
publishDate 2007
url https://ink.library.smu.edu.sg/sis_research/6445
https://ink.library.smu.edu.sg/context/sis_research/article/7448/viewcontent/1282280.1282309.pdf
_version_ 1770575961744474112