CONQUER: Contextual query-aware ranking for video corpus moment retrieval
This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective...
Saved in:
Main Authors: | , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2021
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/6789 https://ink.library.smu.edu.sg/context/sis_research/article/7792/viewcontent/3474085.3475281.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-7792 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-77922022-01-27T09:59:26Z CONQUER: Contextual query-aware ranking for video corpus moment retrieval HOU, Zhijian NGO, Chong-Wah CHAN, W. K. This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is fully engaged in video representation learning, from feature fusion to transformation, the resulting feature is user-centered and has a larger capacity in capturing multi-modal signals specific to query. We conduct studies on two datasets, TVR for closed-world TV episodes and DiDeMo for open-world user-generated videos, to investigate the potential advantages of fusing video and query online as a joint representation for moment retrieval. 2021-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6789 info:doi/10.1145/3474085.3475281 https://ink.library.smu.edu.sg/context/sis_research/article/7792/viewcontent/3474085.3475281.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University cross-modal retrieval moment localization with natural language Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
cross-modal retrieval moment localization with natural language Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
spellingShingle |
cross-modal retrieval moment localization with natural language Artificial Intelligence and Robotics Graphics and Human Computer Interfaces HOU, Zhijian NGO, Chong-Wah CHAN, W. K. CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
description |
This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is fully engaged in video representation learning, from feature fusion to transformation, the resulting feature is user-centered and has a larger capacity in capturing multi-modal signals specific to query. We conduct studies on two datasets, TVR for closed-world TV episodes and DiDeMo for open-world user-generated videos, to investigate the potential advantages of fusing video and query online as a joint representation for moment retrieval. |
format |
text |
author |
HOU, Zhijian NGO, Chong-Wah CHAN, W. K. |
author_facet |
HOU, Zhijian NGO, Chong-Wah CHAN, W. K. |
author_sort |
HOU, Zhijian |
title |
CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
title_short |
CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
title_full |
CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
title_fullStr |
CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
title_full_unstemmed |
CONQUER: Contextual query-aware ranking for video corpus moment retrieval |
title_sort |
conquer: contextual query-aware ranking for video corpus moment retrieval |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2021 |
url |
https://ink.library.smu.edu.sg/sis_research/6789 https://ink.library.smu.edu.sg/context/sis_research/article/7792/viewcontent/3474085.3475281.pdf |
_version_ |
1770576069269651456 |