MMConv: An environment for multimodal conversational search across multiple domains

Although conversational search has become a hot topic in both dialogue research and IR community, the real breakthrough has been limited by the scale and quality of datasets available. To address this fundamental obstacle, we introduce the Multimodal Multi-domain Conversational dataset (MMConv), a f...

Full description

Saved in:
Bibliographic Details
Main Authors: LIAO, Lizi, LONG, Le Hong, ZHANG, Zheng, HUANG, Minlie, CHUA, Tat-Seng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2021
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7286
https://ink.library.smu.edu.sg/context/sis_research/article/8289/viewcontent/3404835.3462970_pvoa.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8289
record_format dspace
spelling sg-smu-ink.sis_research-82892022-09-29T07:44:49Z MMConv: An environment for multimodal conversational search across multiple domains LIAO, Lizi LONG, Le Hong ZHANG, Zheng HUANG, Minlie CHUA, Tat-Seng Although conversational search has become a hot topic in both dialogue research and IR community, the real breakthrough has been limited by the scale and quality of datasets available. To address this fundamental obstacle, we introduce the Multimodal Multi-domain Conversational dataset (MMConv), a fully annotated collection of human-to-human role-playing dialogues spanning over multiple domains and tasks. The contribution is two-fold. First, beyond the task-oriented multimodal dialogues among user and agent pairs, dialogues are fully annotated with dialogue belief states and dialogue acts. More importantly, we create a relatively comprehensive environment for conducting multimodal conversational search with real user settings, structured venue database, annotated image repository as well as crowd-sourced knowledge database. A detailed description of the data collection procedure along with a summary of data structure and analysis is provided. Second, a set of benchmark results for dialogue state tracking, conversational recommendation, response generation as well as a unified model for multiple tasks are reported. We adopt the state-of-the-art methods for these tasks respectively to demonstrate the usability of the data, discuss limitations of current methods and set baselines for future studies. 2021-07-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7286 info:doi/10.1145/3404835.3462970 https://ink.library.smu.edu.sg/context/sis_research/article/8289/viewcontent/3404835.3462970_pvoa.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University datasets multimodal dialogue conversational search Artificial Intelligence and Robotics Numerical Analysis and Scientific Computing Theory and Algorithms
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic datasets
multimodal dialogue
conversational search
Artificial Intelligence and Robotics
Numerical Analysis and Scientific Computing
Theory and Algorithms
spellingShingle datasets
multimodal dialogue
conversational search
Artificial Intelligence and Robotics
Numerical Analysis and Scientific Computing
Theory and Algorithms
LIAO, Lizi
LONG, Le Hong
ZHANG, Zheng
HUANG, Minlie
CHUA, Tat-Seng
MMConv: An environment for multimodal conversational search across multiple domains
description Although conversational search has become a hot topic in both dialogue research and IR community, the real breakthrough has been limited by the scale and quality of datasets available. To address this fundamental obstacle, we introduce the Multimodal Multi-domain Conversational dataset (MMConv), a fully annotated collection of human-to-human role-playing dialogues spanning over multiple domains and tasks. The contribution is two-fold. First, beyond the task-oriented multimodal dialogues among user and agent pairs, dialogues are fully annotated with dialogue belief states and dialogue acts. More importantly, we create a relatively comprehensive environment for conducting multimodal conversational search with real user settings, structured venue database, annotated image repository as well as crowd-sourced knowledge database. A detailed description of the data collection procedure along with a summary of data structure and analysis is provided. Second, a set of benchmark results for dialogue state tracking, conversational recommendation, response generation as well as a unified model for multiple tasks are reported. We adopt the state-of-the-art methods for these tasks respectively to demonstrate the usability of the data, discuss limitations of current methods and set baselines for future studies.
format text
author LIAO, Lizi
LONG, Le Hong
ZHANG, Zheng
HUANG, Minlie
CHUA, Tat-Seng
author_facet LIAO, Lizi
LONG, Le Hong
ZHANG, Zheng
HUANG, Minlie
CHUA, Tat-Seng
author_sort LIAO, Lizi
title MMConv: An environment for multimodal conversational search across multiple domains
title_short MMConv: An environment for multimodal conversational search across multiple domains
title_full MMConv: An environment for multimodal conversational search across multiple domains
title_fullStr MMConv: An environment for multimodal conversational search across multiple domains
title_full_unstemmed MMConv: An environment for multimodal conversational search across multiple domains
title_sort mmconv: an environment for multimodal conversational search across multiple domains
publisher Institutional Knowledge at Singapore Management University
publishDate 2021
url https://ink.library.smu.edu.sg/sis_research/7286
https://ink.library.smu.edu.sg/context/sis_research/article/8289/viewcontent/3404835.3462970_pvoa.pdf
_version_ 1770576304391847936