Large scale online kernel learning

• Main Authors: LU, Jing, HOI, Steven C. H., WANG, Jialei, ZHAO, Peilin, LIU, Zhi-Yong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2016
• Subjects: online learning; kernel method; large scale machine learning; Computer Sciences; Databases and Information Systems; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/3410; https://ink.library.smu.edu.sg/context/sis_research/article/4411/viewcontent/Largescaleonlinekernellearning.pdf
• Institution: Singapore Management University

In this paper, we present a new framework for large scale online kernel learning, making kernel methods efficient and scalable for large-scale online learning applications. Unlike the regular budget online kernel learning scheme that usually uses some budget maintenance strategies to bound the number of support vectors, our framework explores a completely different approach of kernel functional approximation techniques to make the subsequent online learning task efficient and scalable. Specifically, we present two different online kernel machine learning algorithms: (i) Fourier Online Gradient Descent (FOGD) algorithm that applies the random Fourier features for approximating kernel functions; and (ii) Nyström Online Gradient Descent (NOGD) algorithm that applies the Nyström method to approximate large kernel matrices. We explore these two approaches to tackle three online learning tasks: binary classification, multi-class classification, and regression. The encouraging results of our experiments on large-scale datasets validate the effectiveness and efficiency of the proposed algorithms, making them potentially more practical than the family of existing budget online kernel learning approaches.