Simple and optimal stochastic gradient methods for nonsmooth nonconvex optimization

• Main Authors: LI, Zhize, LI, Jian
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: nonconvex optimization; optimal algorithm; proximal gradient descent; variance reduction; local minimum; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8692; https://ink.library.smu.edu.sg/context/sis_research/article/9695/viewcontent/JMLR22_nonsmooth__1_.pdf
• Institution: Singapore Management University

We propose and analyze several stochastic gradient algorithms for finding stationary points or local minimum in nonconvex, possibly with nonsmooth regularizer, finite-sum and online optimization problems. First, we propose a simple proximal stochastic gradient algorithm based on variance reduction c...