An ensemble of epoch-wise empirical Bayes for few-shot learning
Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust...
Saved in:
Main Authors: | , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2020
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/5594 https://ink.library.smu.edu.sg/context/sis_research/article/6597/viewcontent/1904.08479.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-6597 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-65972021-07-05T01:29:41Z An ensemble of epoch-wise empirical Bayes for few-shot learning LIU, Yaoyao SCHIELE, Bernt SUN, Qianru Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust predictions. “Epoch-wise'' means that each training epoch has a Bayes model whose parameters are specifically learned and deployed. ”Empirical'' means that the hyperparameters, e.g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data. We introduce four kinds of hyperprior learners by considering inductive vs. transductive, and epoch-dependent \emph{vs.} epoch-independent, in the paradigm of meta-learning. We conduct extensive experiments for five-class few-shot tasks on three challenging benchmarks: miniImageNet, tieredImageNet, and FC100, and achieve top performance using the epoch-dependent transductive hyperprior learner, which captures the richest information. Our ablation study shows that both “epoch-wise ensemble'' and ”empirical'' encourage high efficiency and robustness in the model performance 2020-08-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/5594 info:doi/10.1007/978-3-030-58517-4_24 https://ink.library.smu.edu.sg/context/sis_research/article/6597/viewcontent/1904.08479.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Confidence predictions Empirical Bayes Empirical Bayes models Hyperparameters Model performance Predictive models Robust predictions Training epochs Artificial Intelligence and Robotics Databases and Information Systems |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Confidence predictions Empirical Bayes Empirical Bayes models Hyperparameters Model performance Predictive models Robust predictions Training epochs Artificial Intelligence and Robotics Databases and Information Systems |
spellingShingle |
Confidence predictions Empirical Bayes Empirical Bayes models Hyperparameters Model performance Predictive models Robust predictions Training epochs Artificial Intelligence and Robotics Databases and Information Systems LIU, Yaoyao SCHIELE, Bernt SUN, Qianru An ensemble of epoch-wise empirical Bayes for few-shot learning |
description |
Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust predictions. “Epoch-wise'' means that each training epoch has a Bayes model whose parameters are specifically learned and deployed. ”Empirical'' means that the hyperparameters, e.g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data. We introduce four kinds of hyperprior learners by considering inductive vs. transductive, and epoch-dependent \emph{vs.} epoch-independent, in the paradigm of meta-learning. We conduct extensive experiments for five-class few-shot tasks on three challenging benchmarks: miniImageNet, tieredImageNet, and FC100, and achieve top performance using the epoch-dependent transductive hyperprior learner, which captures the richest information. Our ablation study shows that both “epoch-wise ensemble'' and ”empirical'' encourage high efficiency and robustness in the model performance |
format |
text |
author |
LIU, Yaoyao SCHIELE, Bernt SUN, Qianru |
author_facet |
LIU, Yaoyao SCHIELE, Bernt SUN, Qianru |
author_sort |
LIU, Yaoyao |
title |
An ensemble of epoch-wise empirical Bayes for few-shot learning |
title_short |
An ensemble of epoch-wise empirical Bayes for few-shot learning |
title_full |
An ensemble of epoch-wise empirical Bayes for few-shot learning |
title_fullStr |
An ensemble of epoch-wise empirical Bayes for few-shot learning |
title_full_unstemmed |
An ensemble of epoch-wise empirical Bayes for few-shot learning |
title_sort |
ensemble of epoch-wise empirical bayes for few-shot learning |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2020 |
url |
https://ink.library.smu.edu.sg/sis_research/5594 https://ink.library.smu.edu.sg/context/sis_research/article/6597/viewcontent/1904.08479.pdf |
_version_ |
1770575521863696384 |