Unifying text, tables, and images for multimodal question answering

• Main Authors: LUO, Haohao, SHEN, Ying, DENG, Yang
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Cross-modal; Input modalities; Language model; Linearisation; Multi-modal; Power; Question Answering; Single-modal; Text format; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9120; https://ink.library.smu.edu.sg/context/sis_research/article/10123/viewcontent/2023.findings_emnlp.626.pdf
• Institution: Singapore Management University

Multimodal question answering (MMQA), which aims to derive the answer from multiple knowledge modalities (e.g., text, tables, and images), has received increasing attention due to its board applications. Current approaches to MMQA often rely on single-modal or bi-modal QA models, which limits their...