Cross-modal recipe retrieval with rich food attributes

Food is rich of visible (e.g., colour, shape) and procedural (e.g., cutting, cooking) attributes. Proper leveraging of these attributes, particularly the interplay among ingredients, cutting and cooking methods, for health-related applications has not been previously explored. This paper investigate...

Full description

Saved in:
Bibliographic Details
Main Authors: CHEN, Jingjing, NGO, Chong-wah, CHUA, Tat-Seng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2017
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6559
https://ink.library.smu.edu.sg/context/sis_research/article/7562/viewcontent/jingjingmm2017.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7562
record_format dspace
spelling sg-smu-ink.sis_research-75622022-01-10T03:35:28Z Cross-modal recipe retrieval with rich food attributes CHEN, Jingjing NGO, Chong-wah CHUA, Tat-Seng Food is rich of visible (e.g., colour, shape) and procedural (e.g., cutting, cooking) attributes. Proper leveraging of these attributes, particularly the interplay among ingredients, cutting and cooking methods, for health-related applications has not been previously explored. This paper investigates cross-modal retrieval of recipes, specifically to retrieve a text-based recipe given a food picture as query. As similar ingredient composition can end up with wildly different dishes depending on the cooking and cutting procedures, the difficulty of retrieval originates from fine-grained recognition of rich attributes from pictures. With a multi-task deep learning model, this paper provides insights on the feasibility of predicting ingredient, cutting and cooking attributes for food recognition and recipe retrieval. In addition, localization of ingredient regions is also possible even when region-level training examples are not provided. Experiment results validate the merit of rich attributes when comparing to the recently proposed ingredient-only retrieval techniques. 2017-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6559 info:doi/10.1145/3123266.3123428 https://ink.library.smu.edu.sg/context/sis_research/article/7562/viewcontent/jingjingmm2017.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Cooking and cutting recognition Cross-modal retrieval Ingredient recognition Recipe retrieval Databases and Information Systems Data Storage Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Cooking and cutting recognition
Cross-modal retrieval
Ingredient recognition
Recipe retrieval
Databases and Information Systems
Data Storage Systems
Graphics and Human Computer Interfaces
spellingShingle Cooking and cutting recognition
Cross-modal retrieval
Ingredient recognition
Recipe retrieval
Databases and Information Systems
Data Storage Systems
Graphics and Human Computer Interfaces
CHEN, Jingjing
NGO, Chong-wah
CHUA, Tat-Seng
Cross-modal recipe retrieval with rich food attributes
description Food is rich of visible (e.g., colour, shape) and procedural (e.g., cutting, cooking) attributes. Proper leveraging of these attributes, particularly the interplay among ingredients, cutting and cooking methods, for health-related applications has not been previously explored. This paper investigates cross-modal retrieval of recipes, specifically to retrieve a text-based recipe given a food picture as query. As similar ingredient composition can end up with wildly different dishes depending on the cooking and cutting procedures, the difficulty of retrieval originates from fine-grained recognition of rich attributes from pictures. With a multi-task deep learning model, this paper provides insights on the feasibility of predicting ingredient, cutting and cooking attributes for food recognition and recipe retrieval. In addition, localization of ingredient regions is also possible even when region-level training examples are not provided. Experiment results validate the merit of rich attributes when comparing to the recently proposed ingredient-only retrieval techniques.
format text
author CHEN, Jingjing
NGO, Chong-wah
CHUA, Tat-Seng
author_facet CHEN, Jingjing
NGO, Chong-wah
CHUA, Tat-Seng
author_sort CHEN, Jingjing
title Cross-modal recipe retrieval with rich food attributes
title_short Cross-modal recipe retrieval with rich food attributes
title_full Cross-modal recipe retrieval with rich food attributes
title_fullStr Cross-modal recipe retrieval with rich food attributes
title_full_unstemmed Cross-modal recipe retrieval with rich food attributes
title_sort cross-modal recipe retrieval with rich food attributes
publisher Institutional Knowledge at Singapore Management University
publishDate 2017
url https://ink.library.smu.edu.sg/sis_research/6559
https://ink.library.smu.edu.sg/context/sis_research/article/7562/viewcontent/jingjingmm2017.pdf
_version_ 1770575987950485504