Knowledge-aware multimodal fashion chatbot

• Main Authors: LIAO, Lizi, ZHOU, You, MA, Yunshan, HONG, Richang, CHUA, Tat-Seng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Artificial Intelligence and Robotics; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7574; https://ink.library.smu.edu.sg/context/sis_research/article/8577/viewcontent/3240508.3241399.pdf
• Institution: Singapore Management University

Multimodal fashion chatbot provides a natural and informative way to fulfill customers’ fashion needs. However, making it ‘smart’ in generating substantive responses remains a challenging problem. In this paper, we present a multimodal domain knowledge enriched fashion chatbot. It forms a taxonomy-based learning module to capture the fine-grained semantics in images and leverages an endto-end neural conversational model to generate responses based on the conversation history, visual semantics, and domain knowledge. To avoid inconsistent dialogues, deep reinforcement learning method is used to further optimize the model.