Zero-shot ingredient recognition by multi-relational graph convolutional network
Recognizing ingredients for a given dish image is at the core of automatic dietary assessment, attracting increasing attention from both industry and academia. Nevertheless, the task is challenging due to the difficulty of collecting and labeling sufficient training data. On one hand, there are hund...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2020
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/6490 https://ink.library.smu.edu.sg/context/sis_research/article/7493/viewcontent/6626_Article_Text_9854_1_10_20200520.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-7493 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-74932022-01-10T05:05:16Z Zero-shot ingredient recognition by multi-relational graph convolutional network CHEN, Jingjing PAN, Liangming WEI, Zhipeng WANG, Xiang NGO, Chong-wah CHUA, Tat-Seng Recognizing ingredients for a given dish image is at the core of automatic dietary assessment, attracting increasing attention from both industry and academia. Nevertheless, the task is challenging due to the difficulty of collecting and labeling sufficient training data. On one hand, there are hundred thousands of food ingredients in the world, ranging from the common to rare. Collecting training samples for all of the ingredient categories is difficult. On the other hand, as the ingredient appearances exhibit huge visual variance during the food preparation, it requires to collect the training samples under different cooking and cutting methods for robust recognition. Since obtaining sufficient fully annotated training data is not easy, a more practical way of scaling up the recognition is to develop models that are capable of recognizing unseen ingredients. Therefore, in this paper, we target the problem of ingredient recognition with zero training samples. More specifically, we introduce multi-relational GCN (graph convolutional network) that integrates ingredient hierarchy, attribute as well as co-occurrence for zero-shot ingredient recognition. Extensive experiments on both Chinese and Japanese food datasets are performed to demonstrate the superior performance of multi-relational GCN and shed light on zero-shot ingredients recognition. 2020-02-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6490 info:doi/10.1609/aaai.v34i07.6626 https://ink.library.smu.edu.sg/context/sis_research/article/7493/viewcontent/6626_Article_Text_9854_1_10_20200520.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Artificial Intelligence and Robotics Graphics and Human Computer Interfaces |
spellingShingle |
Artificial Intelligence and Robotics Graphics and Human Computer Interfaces CHEN, Jingjing PAN, Liangming WEI, Zhipeng WANG, Xiang NGO, Chong-wah CHUA, Tat-Seng Zero-shot ingredient recognition by multi-relational graph convolutional network |
description |
Recognizing ingredients for a given dish image is at the core of automatic dietary assessment, attracting increasing attention from both industry and academia. Nevertheless, the task is challenging due to the difficulty of collecting and labeling sufficient training data. On one hand, there are hundred thousands of food ingredients in the world, ranging from the common to rare. Collecting training samples for all of the ingredient categories is difficult. On the other hand, as the ingredient appearances exhibit huge visual variance during the food preparation, it requires to collect the training samples under different cooking and cutting methods for robust recognition. Since obtaining sufficient fully annotated training data is not easy, a more practical way of scaling up the recognition is to develop models that are capable of recognizing unseen ingredients. Therefore, in this paper, we target the problem of ingredient recognition with zero training samples. More specifically, we introduce multi-relational GCN (graph convolutional network) that integrates ingredient hierarchy, attribute as well as co-occurrence for zero-shot ingredient recognition. Extensive experiments on both Chinese and Japanese food datasets are performed to demonstrate the superior performance of multi-relational GCN and shed light on zero-shot ingredients recognition. |
format |
text |
author |
CHEN, Jingjing PAN, Liangming WEI, Zhipeng WANG, Xiang NGO, Chong-wah CHUA, Tat-Seng |
author_facet |
CHEN, Jingjing PAN, Liangming WEI, Zhipeng WANG, Xiang NGO, Chong-wah CHUA, Tat-Seng |
author_sort |
CHEN, Jingjing |
title |
Zero-shot ingredient recognition by multi-relational graph convolutional network |
title_short |
Zero-shot ingredient recognition by multi-relational graph convolutional network |
title_full |
Zero-shot ingredient recognition by multi-relational graph convolutional network |
title_fullStr |
Zero-shot ingredient recognition by multi-relational graph convolutional network |
title_full_unstemmed |
Zero-shot ingredient recognition by multi-relational graph convolutional network |
title_sort |
zero-shot ingredient recognition by multi-relational graph convolutional network |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2020 |
url |
https://ink.library.smu.edu.sg/sis_research/6490 https://ink.library.smu.edu.sg/context/sis_research/article/7493/viewcontent/6626_Article_Text_9854_1_10_20200520.pdf |
_version_ |
1770575974962823168 |