TRRNet : tiered relation reasoning for compositional visual question answering

• Main Authors: Yang, Xiaofeng, Lin, Guosheng, Lv, Fengmao, Liu, Fayao
• Other Authors: School of Computer Science and Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2020
• Subjects: Engineering::Computer science and engineering; Visual Question Answering; Visual Reasoning
• Online Access: https://hdl.handle.net/10356/144262
• Institution: Nanyang Technological University

Compositional visual question answering requires reasoning over both semantic and geometry object relations. We propose a novel tiered reasoning method that dynamically selects object level candidates based on language representations and generates robust pairwise relations within the selected candidate objects. The proposed tiered relation reasoning method can be compatible with the majority of the existing visual reasoning frameworks, leading to significant performance improvement with very little extra computational cost. Moreover, we propose a policy network that decides the appropriate reasoning steps based on question complexity and current reasoning status. In experiments, our model achieves state-of-the-art performance on two VQA datasets.