PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization

• Main Authors: LI, Zhize, BAO, Hongyan, ZHANG, Xiangliang, RICHTARIK, Peter
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8683; https://ink.library.smu.edu.sg/context/sis_research/article/9686/viewcontent/ICML21_full_page.pdf
• Institution: Singapore Management University

In this paper, we propose a novel stochastic gradient estimator---ProbAbilistic Gradient Estimator (PAGE)---for nonconvex optimization. PAGE is easy to implement as it is designed via a small adjustment to vanilla SGD: in each iteration, PAGE uses the vanilla minibatch SGD update with probability $p...