Meta-transfer learning for few-shot learning

Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural net...

Full description

Saved in:
Bibliographic Details
Main Authors: SUN, Qianru, LIU, Yaoyao, CHUA, Tat-Seng, SCHIELE, Bernt
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/4447
https://ink.library.smu.edu.sg/context/sis_research/article/5450/viewcontent/Sun_Meta_Transfer_Learning_for_Few_Shot_Learning_CVPR_2019_paper.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-5450
record_format dspace
spelling sg-smu-ink.sis_research-54502021-02-19T02:34:22Z Meta-transfer learning for few-shot learning SUN, Qianru LIU, Yaoyao CHUA, Tat-Seng SCHIELE, Bernt Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, meta-learning typically uses shallow neural networks (SNNs), thus limiting its effectiveness. In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks. Specifically, "meta" refers to training multiple tasks, and "transfer" is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum for MTL. We conduct experiments using (5-class, 1-shot) and (5-class, 5-shot) recognition tasks on two challenging few-shot learning benchmarks: miniImageNet and Fewshot-CIFAR100. Extensive comparisons to related works validate that our meta-transfer learning approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy. 2019-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/4447 info:doi/10.1109/CVPR.2019.00049 https://ink.library.smu.edu.sg/context/sis_research/article/5450/viewcontent/Sun_Meta_Transfer_Learning_for_Few_Shot_Learning_CVPR_2019_paper.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Few-shot learning meta-learning hard task mining object classification Artificial Intelligence and Robotics Numerical Analysis and Scientific Computing
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Few-shot learning
meta-learning
hard task mining
object classification
Artificial Intelligence and Robotics
Numerical Analysis and Scientific Computing
spellingShingle Few-shot learning
meta-learning
hard task mining
object classification
Artificial Intelligence and Robotics
Numerical Analysis and Scientific Computing
SUN, Qianru
LIU, Yaoyao
CHUA, Tat-Seng
SCHIELE, Bernt
Meta-transfer learning for few-shot learning
description Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, meta-learning typically uses shallow neural networks (SNNs), thus limiting its effectiveness. In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks. Specifically, "meta" refers to training multiple tasks, and "transfer" is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum for MTL. We conduct experiments using (5-class, 1-shot) and (5-class, 5-shot) recognition tasks on two challenging few-shot learning benchmarks: miniImageNet and Fewshot-CIFAR100. Extensive comparisons to related works validate that our meta-transfer learning approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.
format text
author SUN, Qianru
LIU, Yaoyao
CHUA, Tat-Seng
SCHIELE, Bernt
author_facet SUN, Qianru
LIU, Yaoyao
CHUA, Tat-Seng
SCHIELE, Bernt
author_sort SUN, Qianru
title Meta-transfer learning for few-shot learning
title_short Meta-transfer learning for few-shot learning
title_full Meta-transfer learning for few-shot learning
title_fullStr Meta-transfer learning for few-shot learning
title_full_unstemmed Meta-transfer learning for few-shot learning
title_sort meta-transfer learning for few-shot learning
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/4447
https://ink.library.smu.edu.sg/context/sis_research/article/5450/viewcontent/Sun_Meta_Transfer_Learning_for_Few_Shot_Learning_CVPR_2019_paper.pdf
_version_ 1770574841128157184