Improving interpretable embeddings for ad-hoc video search with generative captions and multi-word concept bank

• Main Authors: WU, Jiaxin, NGO, Chong-wah, CHAN, Wing-Kwong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Ad-hoc video search; Interpretable embedding; Large-scale videotext dataset; Concept bank construction; Out of vocabulary; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9288; https://ink.library.smu.edu.sg/context/sis_research/article/10288/viewcontent/2404.06173v1.pdf
• Institution: Singapore Management University

Aligning a user query and video clips in cross-modal latent space and that with semantic concepts are two mainstream approaches for ad-hoc video search (AVS). However, the effectiveness of existing approaches is bottlenecked by the small sizes of available video-text datasets and the low quality of...