Deep-based ingredient recognition for cooking recipe retrieval
Retrieving recipes corresponding to given dish pictures facilitates the estimation of nutrition facts, which is crucial to various health relevant applications. The current approaches mostly focus on recognition of food category based on global dish appearance without explicit analysis of ingredient...
Saved in:
Main Authors: | , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2016
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/6498 https://ink.library.smu.edu.sg/context/sis_research/article/7501/viewcontent/2964284.2964315.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-7501 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-75012022-01-10T04:58:04Z Deep-based ingredient recognition for cooking recipe retrieval CHEN, Jingjing NGO, Chong-wah Retrieving recipes corresponding to given dish pictures facilitates the estimation of nutrition facts, which is crucial to various health relevant applications. The current approaches mostly focus on recognition of food category based on global dish appearance without explicit analysis of ingredient composition. Such approaches are incapable for retrieval of recipes with unknown food categories, a problem referred to as zero-shot retrieval. On the other hand, content-based retrieval without knowledge of food categories is also difficult to attain satisfactory performance due to large visual variations in food appearance and ingredient composition. As the number of ingredients is far less than food categories, understanding ingredients underlying dishes in principle is more scalable than recognizing every food category and thus is suitable for zero-shot retrieval. Nevertheless, ingredient recognition is a task far harder than food categorization, and this seriously challenges the feasibility of relying on them for retrieval. This paper proposes deep architectures for simultaneous learning of ingredient recognition and food categorization, by exploiting the mutual but also fuzzy relationship between them. The learnt deep features and semantic labels of ingredients are then innovatively applied for zero-shot retrieval of recipes. By experimenting on a large Chinese food dataset with images of highly complex dish appearance, this paper demonstrates the feasibility of ingredient recognition and sheds light on this zero-shot problem peculiar to cooking recipe retrieval. 2016-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6498 info:doi/10.1145/2964284.2964315 https://ink.library.smu.edu.sg/context/sis_research/article/7501/viewcontent/2964284.2964315.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Food categorization Ingredient recognition Multitask deep learning Zero-shot retrieval Databases and Information Systems Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Food categorization Ingredient recognition Multitask deep learning Zero-shot retrieval Databases and Information Systems Graphics and Human Computer Interfaces |
spellingShingle |
Food categorization Ingredient recognition Multitask deep learning Zero-shot retrieval Databases and Information Systems Graphics and Human Computer Interfaces CHEN, Jingjing NGO, Chong-wah Deep-based ingredient recognition for cooking recipe retrieval |
description |
Retrieving recipes corresponding to given dish pictures facilitates the estimation of nutrition facts, which is crucial to various health relevant applications. The current approaches mostly focus on recognition of food category based on global dish appearance without explicit analysis of ingredient composition. Such approaches are incapable for retrieval of recipes with unknown food categories, a problem referred to as zero-shot retrieval. On the other hand, content-based retrieval without knowledge of food categories is also difficult to attain satisfactory performance due to large visual variations in food appearance and ingredient composition. As the number of ingredients is far less than food categories, understanding ingredients underlying dishes in principle is more scalable than recognizing every food category and thus is suitable for zero-shot retrieval. Nevertheless, ingredient recognition is a task far harder than food categorization, and this seriously challenges the feasibility of relying on them for retrieval. This paper proposes deep architectures for simultaneous learning of ingredient recognition and food categorization, by exploiting the mutual but also fuzzy relationship between them. The learnt deep features and semantic labels of ingredients are then innovatively applied for zero-shot retrieval of recipes. By experimenting on a large Chinese food dataset with images of highly complex dish appearance, this paper demonstrates the feasibility of ingredient recognition and sheds light on this zero-shot problem peculiar to cooking recipe retrieval. |
format |
text |
author |
CHEN, Jingjing NGO, Chong-wah |
author_facet |
CHEN, Jingjing NGO, Chong-wah |
author_sort |
CHEN, Jingjing |
title |
Deep-based ingredient recognition for cooking recipe retrieval |
title_short |
Deep-based ingredient recognition for cooking recipe retrieval |
title_full |
Deep-based ingredient recognition for cooking recipe retrieval |
title_fullStr |
Deep-based ingredient recognition for cooking recipe retrieval |
title_full_unstemmed |
Deep-based ingredient recognition for cooking recipe retrieval |
title_sort |
deep-based ingredient recognition for cooking recipe retrieval |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2016 |
url |
https://ink.library.smu.edu.sg/sis_research/6498 https://ink.library.smu.edu.sg/context/sis_research/article/7501/viewcontent/2964284.2964315.pdf |
_version_ |
1770575976808316928 |