Hybrid stochastic-deterministic minibatch proximal gradient: Less-than-single-pass optimization with nearly optimal generalization

• Main Authors: ZHOU, Pan, YUAN, Xiaotong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/9030; https://ink.library.smu.edu.sg/context/sis_research/article/10033/viewcontent/2020_ICML_HSDN.pdf
• Institution: Singapore Management University

Stochastic variance-reduced gradient (SVRG) algorithms have been shown to work favorably in solving large-scale learning problems. Despite the remarkable success, the stochastic gradient complexity of SVRG-type algorithms usually scales linearly with data size and thus could still be expensive for h...