Multimodal review generation for recommender systems

Key to recommender systems is learning user preferences, which are expressed through various modalities. In online reviews, for instance, this manifests in numerical rating, textual content, as well as visual images. In this work, we hypothesize that modelling these modalities jointly would result i...

Full description

Saved in:
Bibliographic Details
Main Authors: TRUONG, Quoc Tuan, LAUW, Hady W.
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/4388
https://ink.library.smu.edu.sg/context/sis_research/article/5391/viewcontent/p1864_truong.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-5391
record_format dspace
spelling sg-smu-ink.sis_research-53912020-04-03T09:25:52Z Multimodal review generation for recommender systems TRUONG, Quoc Tuan LAUW, Hady W. Key to recommender systems is learning user preferences, which are expressed through various modalities. In online reviews, for instance, this manifests in numerical rating, textual content, as well as visual images. In this work, we hypothesize that modelling these modalities jointly would result in a more holistic representation of a review towards more accurate recommendations. Therefore, we propose Multimodal Review Generation (MRG), a neural approach that simultaneously models a rating prediction component and a review text generation component. We hypothesize that the shared user and item representations would augment the rating prediction with richer information from review text, while sensitizing the generated review text to sentiment features based on user and item of interest. Moreover, when review photos are available, visual features could inform the review text generation further. Comprehensive experiments on real-life datasets from several major US cities show that the proposed model outperforms comparable multimodal baselines, while an ablation analysis establishes the relative contributions of the respective components of the joint model. 2019-05-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/4388 info:doi/10.1145/3308558.3313463 https://ink.library.smu.edu.sg/context/sis_research/article/5391/viewcontent/p1864_truong.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Databases and Information Systems Numerical Analysis and Scientific Computing
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Databases and Information Systems
Numerical Analysis and Scientific Computing
spellingShingle Databases and Information Systems
Numerical Analysis and Scientific Computing
TRUONG, Quoc Tuan
LAUW, Hady W.
Multimodal review generation for recommender systems
description Key to recommender systems is learning user preferences, which are expressed through various modalities. In online reviews, for instance, this manifests in numerical rating, textual content, as well as visual images. In this work, we hypothesize that modelling these modalities jointly would result in a more holistic representation of a review towards more accurate recommendations. Therefore, we propose Multimodal Review Generation (MRG), a neural approach that simultaneously models a rating prediction component and a review text generation component. We hypothesize that the shared user and item representations would augment the rating prediction with richer information from review text, while sensitizing the generated review text to sentiment features based on user and item of interest. Moreover, when review photos are available, visual features could inform the review text generation further. Comprehensive experiments on real-life datasets from several major US cities show that the proposed model outperforms comparable multimodal baselines, while an ablation analysis establishes the relative contributions of the respective components of the joint model.
format text
author TRUONG, Quoc Tuan
LAUW, Hady W.
author_facet TRUONG, Quoc Tuan
LAUW, Hady W.
author_sort TRUONG, Quoc Tuan
title Multimodal review generation for recommender systems
title_short Multimodal review generation for recommender systems
title_full Multimodal review generation for recommender systems
title_fullStr Multimodal review generation for recommender systems
title_full_unstemmed Multimodal review generation for recommender systems
title_sort multimodal review generation for recommender systems
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/4388
https://ink.library.smu.edu.sg/context/sis_research/article/5391/viewcontent/p1864_truong.pdf
_version_ 1770574694831882240