Multi modal personalized explanation generation

• Main Author: Marantika, Winda Kirana
• Other Authors: Alex Chichung Kot
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Engineering; Recommender; NLP; VQVAE; Personalized explanation; Transformer
• Online Access: https://hdl.handle.net/10356/175910
• Institution: Nanyang Technological University

In the domain of recommendation systems, personalized explainable recommendation systems are gaining significant attention. This dissertation contributes to this field by compiling publicly available real-world image data for the TripAdvisor dataset. Furthermore, a method has been developed and tested that generates these recommendations by incorporating image data, user ID, item ID, user persona, and item persona. The image feature data is derived from a quantized vector encoder from VQVAE. We have devised and tested a method capable of generating personalized explainable recommendations by incorporating image data as input. The visual token is extracted from the output of a quantized vector encoder from VQVAE. This data is combined with the user ID, item ID, user persona, and item persona to generate personalized explanations through a lightweight encoder-decoder transformer. The evaluation concludes that our model can generate superior text explainability, diversity, and quality on two publicly available datasets if no feature word is incorporated in the training or evaluation process. The addition of an image feature can enhance the model’s ability to generate better text quality.