Learning with multiple kernels : algorithms and applications

• Main Author: Hao, Xiao
• Other Authors: Hoi Chu Hong
• Format: Theses and Dissertations
• Language: English
• Published: 2014
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems::Information storage and retrieval; DRNTU::Engineering::Computer science and engineering::Information systems::Information systems applications
• Online Access: https://hdl.handle.net/10356/61755
• Institution: Nanyang Technological University

Multiple kernel learning (MKL) is a family of machine learning techniques by learning with multiple kernel functions/matrices. It has been demonstrated as a promising technique for solving many real-world applications, especially for its power of exploiting multiple kernels for fusing diverse information from multiple sources. Beyond the traditional MKL studies for classification, this thesis investigates a more general research problem of learning with multiple kernels for different applications. Specifically, thesis has proposed a family of novel algorithms for learning with multiple kernels to tackle classification, hashing and similarity learning tasks, respectively. The following gives the main contributions of the proposed techniques. Firstly, we aim to investigate novel learning with multiple kernels techniques for classification tasks. Specifically, this thesis presents a novel framework of Multiple Kernel Boosting (MKBoost) for classification tasks, which applies the idea of boosting techniques to learn kernel-based classifiers with multiple kernels for resolving classification problems. Conventional MKL methods often formulate the problem as an optimization task of learning the optimal combinations of both kernels and classifiers, which usually results in some forms of challenging optimization tasks that are often difficult to be solved. Different from the existing MKL methods, in this thesis, we investigate a boosting framework of multiple kernel learning for classification tasks, i.e., we adopt boosting to solve a variant of MKL problem, which avoids solving the complicated optimization tasks. Based on the proposed framework, we propose several variants of MKBoost algorithms and extensively examine their empirical performance on a number of benchmark datasets in comparisons to various state-of-the-art MKL algorithms on classification tasks. Experimental results show that the proposed method is more effective and efficient than the existing MKL techniques. Secondly, we aim to investigate novel learning with multiple kernels techniques for hashing tasks in multimedia similarity search. Specifically, this thesis proposes a Boosting Multi-Kernel Locality-Sensitive Hashing (BMKLSH) scheme that significantly boosts the retrieval performance of existing hashing methods by making use of multiple kernels. Similarity search is a key challenge for multimedia retrieval applications where data are usually represented in high-dimensional space. Among various algorithms proposed for similarity search in high-dimensional space, Locality-Sensitive Hashing (LSH) is the most popular one, which recently has been extended to Kernelized Locality-Sensitive Hashing (KLSH) by exploiting kernel similarity for better retrieval efficacy. Typically, KLSH works only with a single kernel, which is often limited in real-world multimedia applications, where data may originate from multiple resources or can be represented in several different forms. For example, in content-based multimedia retrieval, a variety of features can be extracted to represent contents of an image. So BMKLSH is proposed to overcome the limitation of regular KLSH. We conduct extensive experiments for large-scale content-based image retrieval, in which encouraging results show that the proposed method outperforms the state-of-the-art techniques. Finally, we aim to investigate novel learning with multiple kernels techniques for learning similarity functions in multimedia retrieval. This thesis proposes a novel Online Multiple Kernel Similarity (OMKS) learning method, which learns a flexible nonlinear proximity function with multiple kernels to improve visual similarity search in CBIR. Recent years have witnessed a number of studies on distance metric learning to improve visual similarity search in CBIR. Despite their successes, most existing methods on distance metric learning are limited in two aspects. First, they usually assume the target proximity function follows the family of Mahalanobis distances, which limits their capacity of measuring similarity of complex patterns in real applications. Second, they often cannot effectively handle the similarity measure of multi-modal data that may originate from multiple resources. To overcome these limitations, this thesis investigates an online kernel similarity learning framework for learning kernel-based proximity functions, which goes beyond the conventional linear distance metric learning approaches. Based on the framework, OMKS is proposed. We evaluate the proposed technique for CBIR on a variety of image data sets, in which encouraging results show that OMKS outperforms the state-of-the-art techniques significantly.