Online Kernel Selection: Algorithms and Evaluations

Kernel methods have been successfully applied to many machine learning problems. Nevertheless, since the performance of kernel methods depends heavily on the type of kernels being used, identifying good kernels among a set of given kernels is important to the success of kernel methods. A straightfor...

Full description

Saved in:
Bibliographic Details
Main Authors: YANG, Tianbao, MAHDAVI, Mehrdad, JIN, Rong, YI, Jinfeng, HOI, Steven C. H.
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2012
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/2344
https://ink.library.smu.edu.sg/context/sis_research/article/3344/viewcontent/Online_Kernel_Selection_Algorithms_and_Evaluations.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-3344
record_format dspace
spelling sg-smu-ink.sis_research-33442018-12-03T03:34:39Z Online Kernel Selection: Algorithms and Evaluations YANG, Tianbao MAHDAVI, Mehrdad JIN, Rong YI, Jinfeng HOI, Steven C. H. Kernel methods have been successfully applied to many machine learning problems. Nevertheless, since the performance of kernel methods depends heavily on the type of kernels being used, identifying good kernels among a set of given kernels is important to the success of kernel methods. A straightforward approach to address this problem is cross-validation by training a separate classifier for each kernel and choosing the best kernel classifier out of them. Another approach is Multiple Kernel Learning (MKL), which aims to learn a single kernel classifier from an optimal combination of multiple kernels. However, both approaches suffer from a high computational cost in computing the full kernel matrices and in training, especially when the number of kernels or the number of training examples is very large. In this paper, we tackle this problem by proposing an efficient online kernel selection algorithm. It incrementally learns a weight for each kernel classifier. The weight for each kernel classifier can help us to select a good kernel among a set of given kernels. The proposed approach is efficient in that (i) it is an online approach and therefore avoids computing all the full kernel matrices before training; (ii) it only updates a single kernel classifier each time by a sampling technique and therefore saves time on updating kernel classifiers with poor performance; (iii) it has a theoretically guaranteed performance compared to the best kernel predictor. Empirical studies on image classification tasks demonstrate the effectiveness of the proposed approach for selecting a good kernel among a set of kernels. 2012-07-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/2344 https://ink.library.smu.edu.sg/context/sis_research/article/3344/viewcontent/Online_Kernel_Selection_Algorithms_and_Evaluations.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Computer Sciences Databases and Information Systems Theory and Algorithms
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Computer Sciences
Databases and Information Systems
Theory and Algorithms
spellingShingle Computer Sciences
Databases and Information Systems
Theory and Algorithms
YANG, Tianbao
MAHDAVI, Mehrdad
JIN, Rong
YI, Jinfeng
HOI, Steven C. H.
Online Kernel Selection: Algorithms and Evaluations
description Kernel methods have been successfully applied to many machine learning problems. Nevertheless, since the performance of kernel methods depends heavily on the type of kernels being used, identifying good kernels among a set of given kernels is important to the success of kernel methods. A straightforward approach to address this problem is cross-validation by training a separate classifier for each kernel and choosing the best kernel classifier out of them. Another approach is Multiple Kernel Learning (MKL), which aims to learn a single kernel classifier from an optimal combination of multiple kernels. However, both approaches suffer from a high computational cost in computing the full kernel matrices and in training, especially when the number of kernels or the number of training examples is very large. In this paper, we tackle this problem by proposing an efficient online kernel selection algorithm. It incrementally learns a weight for each kernel classifier. The weight for each kernel classifier can help us to select a good kernel among a set of given kernels. The proposed approach is efficient in that (i) it is an online approach and therefore avoids computing all the full kernel matrices before training; (ii) it only updates a single kernel classifier each time by a sampling technique and therefore saves time on updating kernel classifiers with poor performance; (iii) it has a theoretically guaranteed performance compared to the best kernel predictor. Empirical studies on image classification tasks demonstrate the effectiveness of the proposed approach for selecting a good kernel among a set of kernels.
format text
author YANG, Tianbao
MAHDAVI, Mehrdad
JIN, Rong
YI, Jinfeng
HOI, Steven C. H.
author_facet YANG, Tianbao
MAHDAVI, Mehrdad
JIN, Rong
YI, Jinfeng
HOI, Steven C. H.
author_sort YANG, Tianbao
title Online Kernel Selection: Algorithms and Evaluations
title_short Online Kernel Selection: Algorithms and Evaluations
title_full Online Kernel Selection: Algorithms and Evaluations
title_fullStr Online Kernel Selection: Algorithms and Evaluations
title_full_unstemmed Online Kernel Selection: Algorithms and Evaluations
title_sort online kernel selection: algorithms and evaluations
publisher Institutional Knowledge at Singapore Management University
publishDate 2012
url https://ink.library.smu.edu.sg/sis_research/2344
https://ink.library.smu.edu.sg/context/sis_research/article/3344/viewcontent/Online_Kernel_Selection_Algorithms_and_Evaluations.pdf
_version_ 1770572104639447040