Unified locally linear classifiers with diversity-promoting anchor points

Locally Linear Support Vector Machine (LLSVM) has been actively used in classification tasks due to its capability of classifying nonlinear patterns. However, existing LLSVM suffers from two drawbacks: (1) a particular and appropriate regularization for LLSVM has not yet been addressed; (2) it usual...

Full description

Saved in:
Bibliographic Details
Main Authors: LIU, Chenghao, ZHANG, Teng, ZHAO, Peilin, SUN, Jianling, HOI, Steven C. H.
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2018
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/4183
https://ink.library.smu.edu.sg/context/sis_research/article/5186/viewcontent/17037_76708_1_PB.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-5186
record_format dspace
spelling sg-smu-ink.sis_research-51862020-03-25T03:23:14Z Unified locally linear classifiers with diversity-promoting anchor points LIU, Chenghao ZHANG, Teng ZHAO, Peilin SUN, Jianling HOI, Steven C. H. Locally Linear Support Vector Machine (LLSVM) has been actively used in classification tasks due to its capability of classifying nonlinear patterns. However, existing LLSVM suffers from two drawbacks: (1) a particular and appropriate regularization for LLSVM has not yet been addressed; (2) it usually adopts a three-stage learning scheme composed of learning anchor points by clustering, learning local coding coordinates by a predefined coding scheme, and finally learning for training classifiers. We argue that this decoupled approaches oversimplifies the original optimization problem, resulting in a large deviation due to the disparate purpose of each step. To address the first issue, we propose a novel diversified regularization which could capture infrequent patterns and reduce the model size without sacrificing the representation power. Based on this regularization, we develop a joint optimization algorithm among anchor points, local coding coordinates and classifiers to simultaneously minimize the overall classification risk, which is termed as Diversified and Unified Locally Linear Support Vector Machine (DU-LLSVM for short). To the best of our knowledge, DU-LLSVM is the first principled method that directly learns sparse local coding and can be easily generalized to other supervised learning models.Extensive experiments showed that DU-LLSVM consistently surpassed several state-of-the-art methods with a predefined local coding scheme (e.g. LLSVM) or a supervised anchor point learning (e.g. SAPL-LLSVM). 2018-02-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/4183 https://ink.library.smu.edu.sg/context/sis_research/article/5186/viewcontent/17037_76708_1_PB.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Classification Manifold Learning Support Vector Machine Databases and Information Systems Numerical Analysis and Scientific Computing
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Classification
Manifold Learning
Support Vector Machine
Databases and Information Systems
Numerical Analysis and Scientific Computing
spellingShingle Classification
Manifold Learning
Support Vector Machine
Databases and Information Systems
Numerical Analysis and Scientific Computing
LIU, Chenghao
ZHANG, Teng
ZHAO, Peilin
SUN, Jianling
HOI, Steven C. H.
Unified locally linear classifiers with diversity-promoting anchor points
description Locally Linear Support Vector Machine (LLSVM) has been actively used in classification tasks due to its capability of classifying nonlinear patterns. However, existing LLSVM suffers from two drawbacks: (1) a particular and appropriate regularization for LLSVM has not yet been addressed; (2) it usually adopts a three-stage learning scheme composed of learning anchor points by clustering, learning local coding coordinates by a predefined coding scheme, and finally learning for training classifiers. We argue that this decoupled approaches oversimplifies the original optimization problem, resulting in a large deviation due to the disparate purpose of each step. To address the first issue, we propose a novel diversified regularization which could capture infrequent patterns and reduce the model size without sacrificing the representation power. Based on this regularization, we develop a joint optimization algorithm among anchor points, local coding coordinates and classifiers to simultaneously minimize the overall classification risk, which is termed as Diversified and Unified Locally Linear Support Vector Machine (DU-LLSVM for short). To the best of our knowledge, DU-LLSVM is the first principled method that directly learns sparse local coding and can be easily generalized to other supervised learning models.Extensive experiments showed that DU-LLSVM consistently surpassed several state-of-the-art methods with a predefined local coding scheme (e.g. LLSVM) or a supervised anchor point learning (e.g. SAPL-LLSVM).
format text
author LIU, Chenghao
ZHANG, Teng
ZHAO, Peilin
SUN, Jianling
HOI, Steven C. H.
author_facet LIU, Chenghao
ZHANG, Teng
ZHAO, Peilin
SUN, Jianling
HOI, Steven C. H.
author_sort LIU, Chenghao
title Unified locally linear classifiers with diversity-promoting anchor points
title_short Unified locally linear classifiers with diversity-promoting anchor points
title_full Unified locally linear classifiers with diversity-promoting anchor points
title_fullStr Unified locally linear classifiers with diversity-promoting anchor points
title_full_unstemmed Unified locally linear classifiers with diversity-promoting anchor points
title_sort unified locally linear classifiers with diversity-promoting anchor points
publisher Institutional Knowledge at Singapore Management University
publishDate 2018
url https://ink.library.smu.edu.sg/sis_research/4183
https://ink.library.smu.edu.sg/context/sis_research/article/5186/viewcontent/17037_76708_1_PB.pdf
_version_ 1770574395609186304