SSRGD: Simple Stochastic Recursive Gradient Descent for escaping saddle points

• 主要作者: LI, Zhize
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2019
• 主題: Databases and Information Systems
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/8679; https://ink.library.smu.edu.sg/context/sis_research/article/9682/viewcontent/NeurIPS_2019_ssrgd_simple_stochastic_recursive_gradient_descent_for_escaping_saddle_points_Paper.pdf
• 機構: Singapore Management University

We analyze stochastic gradient algorithms for optimizing nonconvex problems. In particular, our goal is to find local minima (second-order stationary points) instead of just finding first-order stationary points which may be some bad unstable saddle points. We show that a simple perturbed version of...