Interventional few-shot learning
We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowl...
Saved in:
Main Authors: | , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2020
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/5596 https://ink.library.smu.edu.sg/context/sis_research/article/6599/viewcontent/NeurIPS_2020_interventional_few_shot_learning_Paper.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-6599 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-65992021-07-05T02:44:49Z Interventional few-shot learning YUE, Zhongqi ZHANG Hanwang, SUN, Qianru HUA, Xian-Sheng We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowledge, sample features, and labels. Thanks to it, we propose a novel FSL paradigm: Interventional Few-Shot Learning (IFSL). Specifically, we develop three effective IFSL algorithmic implementations based on the backdoor adjustment, which is essentially a causal intervention towards the SCM of many-shot learning: the upper-bound of FSL in a causal view. It is worth noting that the contribution of IFSL is orthogonal to existing fine-tuning and meta-learning based FSL methods, hence IFSL can improve all of them, achieving a new 1-/5-shot state-of-the-art on miniImageNet, tieredImageNet, and cross-domain CUB. Code is released at https://github.com/yue-zhongqi/ifsl. 2020-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/5596 https://ink.library.smu.edu.sg/context/sis_research/article/6599/viewcontent/NeurIPS_2020_interventional_few_shot_learning_Paper.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Causal intervention Causal model Cross-domain Fine tuning Interventional Metalearning Sample features State of the art Artificial Intelligence and Robotics Databases and Information Systems |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Causal intervention Causal model Cross-domain Fine tuning Interventional Metalearning Sample features State of the art Artificial Intelligence and Robotics Databases and Information Systems |
spellingShingle |
Causal intervention Causal model Cross-domain Fine tuning Interventional Metalearning Sample features State of the art Artificial Intelligence and Robotics Databases and Information Systems YUE, Zhongqi ZHANG Hanwang, SUN, Qianru HUA, Xian-Sheng Interventional few-shot learning |
description |
We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowledge, sample features, and labels. Thanks to it, we propose a novel FSL paradigm: Interventional Few-Shot Learning (IFSL). Specifically, we develop three effective IFSL algorithmic implementations based on the backdoor adjustment, which is essentially a causal intervention towards the SCM of many-shot learning: the upper-bound of FSL in a causal view. It is worth noting that the contribution of IFSL is orthogonal to existing fine-tuning and meta-learning based FSL methods, hence IFSL can improve all of them, achieving a new 1-/5-shot state-of-the-art on miniImageNet, tieredImageNet, and cross-domain CUB. Code is released at https://github.com/yue-zhongqi/ifsl. |
format |
text |
author |
YUE, Zhongqi ZHANG Hanwang, SUN, Qianru HUA, Xian-Sheng |
author_facet |
YUE, Zhongqi ZHANG Hanwang, SUN, Qianru HUA, Xian-Sheng |
author_sort |
YUE, Zhongqi |
title |
Interventional few-shot learning |
title_short |
Interventional few-shot learning |
title_full |
Interventional few-shot learning |
title_fullStr |
Interventional few-shot learning |
title_full_unstemmed |
Interventional few-shot learning |
title_sort |
interventional few-shot learning |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2020 |
url |
https://ink.library.smu.edu.sg/sis_research/5596 https://ink.library.smu.edu.sg/context/sis_research/article/6599/viewcontent/NeurIPS_2020_interventional_few_shot_learning_Paper.pdf |
_version_ |
1770575522790637568 |