Contrastive video question answering via video graph transformer

• Main Authors: XIAO, Junbin Xiao, ZHOU, Pan, YAO, Angela, LI, Yicong, HONG, Richang, YAN, Shuicheng, CHUA, Tat-Seng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: VideoQA; Cross-Modal Visual Reasoning; Video-Language; Dynamic Visual Graphs; Contrastive Learning; Transformer; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9053; https://ink.library.smu.edu.sg/context/sis_research/article/10056/viewcontent/2023_TPAMI_ContrastiveVideo.pdf
• Institution: Singapore Management University

We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT’s uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their rel...