Structured and natural responses co-generation for conversational search

• Main Authors: YE, Chenchen, LIAO, Lizi, FENG, Fuli, JI, Wei, CHUA, Tat-Seng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Bidirectional association; Co-generation; Conversational search; Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/7720; https://ink.library.smu.edu.sg/context/sis_research/article/8723/viewcontent/Structured_and_natural_responses_co_generation_for_conversational_search.pdf
• Institution: Singapore Management University

Generating fluent and informative natural responses while main- taining representative internal states for search optimization is critical for conversational search systems. Existing approaches ei- ther 1) predict structured dialog acts first and then generate natural response; or 2) map conversation context to natural responses di- rectly in an end-to-end manner. Both kinds of approaches have shortcomings. The former suffers from error accumulation while the semantic associations between structured acts and natural re- sponses are confined in single direction. The latter emphasizes generating natural responses but fails to predict structured acts. Therefore, we propose a neural co-generation model that gener- ates the two concurrently. The key lies in a shared latent space shaped by two informed priors. Specifically, we design structured dialog acts and natural response auto-encoding as two auxiliary tasks in an interconnected network architecture. It allows for the concurrent generation and bidirectional semantic associations. The shared latent space also enables asynchronous reinforcement learn- ing for further joint optimization. Experiments show that our model achieves significant performance improvements.