Hierarchical multimodal attention for end-to-end audio-visual scene-aware dialogue response generation

• Main Authors: LE, Hung, SAHOO, Doyen, CHEN, Nancy F., HOI, Steven C. H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: Audio-visual scene-aware dialogue; Dialogue system; Multimodal attention; Neural network; Response generation; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/5259; https://ink.library.smu.edu.sg/context/sis_research/article/6262/viewcontent/Hierarchical_multimodal_attention_av.pdf
• Institution: Singapore Management University

This work is extended from our participation in the Dialogue System Technology Challenge (DSTC7), where we participated in the Audio Visual Scene-aware Dialogue System (AVSD) track. The AVSD track evaluates how dialogue systems understand video scenes and responds to users about the video visual and...