Projection-free distributed online learning in networks

• Main Authors: ZHANG, Wenpeng, ZHAO, Peilin, ZHU, Wenwu, HOI, Steven C. H., ZHANG, Tong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/3970; https://ink.library.smu.edu.sg/context/sis_research/article/4972/viewcontent/26._Oct05_2017___Projection_free_Distributed_Online_Learning_in_Networks__ICML2017_.pdf
• Institution: Singapore Management University

The conditional gradient algorithm has regained a surge of research interest in recent years due to its high efficiency in handling large-scale machine learning problems. However, none of existing studies has explored it in the distributed online learning setting, where locally light computation is assumed. In this paper, we fill this gap by proposing the distributed online conditional gradient algorithm, which eschews the expensive projection operation needed in its counterpart algorithms by exploiting much simpler linear optimization steps. We give a regret bound for the proposed algorithm as a function of the network size and topology, which will be smaller on smaller graphs or ”well-connected” graphs. Experiments on two large-scale real-world datasets for a multiclass classification task confirm the computational benefit of the proposed algorithm and also verify the theoretical regret bound.