PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization

• المؤلفون الرئيسيون: LI, Zhize, BAO, Hongyan, ZHANG, Xiangliang, RICHTARIK, Peter
• التنسيق: text
• اللغة: English
• منشور في: Institutional Knowledge at Singapore Management University 2021
• الموضوعات: Databases and Information Systems
• الوصول للمادة أونلاين: https://ink.library.smu.edu.sg/sis_research/8683; https://ink.library.smu.edu.sg/context/sis_research/article/9686/viewcontent/ICML21_full_page.pdf

In this paper, we propose a novel stochastic gradient estimator---ProbAbilistic Gradient Estimator (PAGE)---for nonconvex optimization. PAGE is easy to implement as it is designed via a small adjustment to vanilla SGD: in each iteration, PAGE uses the vanilla minibatch SGD update with probability $p...