PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization

• Main Authors: LI, Zhize, BAO, Hongyan, ZHANG, Xiangliang, RICHTARIK, Peter
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2021
• 主題: Databases and Information Systems
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/8683; https://ink.library.smu.edu.sg/context/sis_research/article/9686/viewcontent/ICML21_full_page.pdf
• 機構: Singapore Management University

In this paper, we propose a novel stochastic gradient estimator---ProbAbilistic Gradient Estimator (PAGE)---for nonconvex optimization. PAGE is easy to implement as it is designed via a small adjustment to vanilla SGD: in each iteration, PAGE uses the vanilla minibatch SGD update with probability $p...