A hybrid stochastic-deterministic minibatch proximal gradient method for efficient optimization and generalization

• Main Authors: ZHOU, Pan, YUAN, Xiao-Tong, LIN Zhouchen, HOI, Steven C. H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Convex Optimization; Precondition; Online Convex Optimization; Stochastic Variance-Reduced Algorithm; Artificial Intelligence and Robotics; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/8979; https://ink.library.smu.edu.sg/context/sis_research/article/9982/viewcontent/2021_TPAMI_HSDN.pdf
• Institution: Singapore Management University

Despite the success of stochastic variance-reduced gradient (SVRG) algorithms in solving large-scale problems, their stochastic gradient complexity often scales linearly with data size and is expensive for huge data. Accordingly, we propose a hybrid stochastic-deterministic minibatch proximal gradie...