Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization

Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization

Variance reduction techniques like SVRG provide simple and fast algorithms for optimizing a convex finite-sum objective. For nonconvex objectives, these techniques can also find a first-order stationary point (with small gradient). However, in nonconvex optimization it is often crucial to find a sec...

Saved in:

書目詳細資料
Main Authors:	GE, Rong, LI, Zhize, WANG, Weiyao, WANG, Xiang
格式:	text
語言:	English
出版:	Institutional Knowledge at Singapore Management University 2019
主題:	Databases and Information Systems
在線閱讀:	https://ink.library.smu.edu.sg/sis_research/8677 https://ink.library.smu.edu.sg/context/sis_research/article/9680/viewcontent/COLT19_stabilizedsvrg.pdf
標簽:	添加標簽沒有標簽, 成為第一個標記此記錄!

相似書籍

A simple proximal stochastic gradient method for nonsmooth nonconvex optimization
由: LI, Zhize, et al.
出版: (2018)

PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization
由: LI, Zhize, et al.
出版: (2021)

Simple and optimal stochastic gradient methods for nonsmooth nonconvex optimization
由: LI, Zhize, et al.
出版: (2022)

BEER: Fast O(1/T) rate for decentralized nonconvex optimization with communication compression
由: ZHAO, Haoyu, et al.
出版: (2022)

DESTRESS: Computation-optimal and communication-efficient decentralized nonconvex finite-sum optimization
由: LI, Boyue, et al.
出版: (2022)

A unified variance-reduced accelerated gradient method for convex optimization
由: LAN, Guanghui, et al.
出版: (2019)

SSRGD: Simple Stochastic Recursive Gradient Descent for escaping saddle points
由: LI, Zhize
出版: (2019)

Stochastic gradient Hamiltonian Monte Carlo with variance reduction for Bayesian inference
由: LI, Zhize, et al.
出版: (2019)

Faster rates for compressed federated learning with client-variance reduction
由: ZHAO, Haoyu, et al.
出版: (2024)

A fast Anderson-Chebyshev acceleration for nonlinear optimization
由: LI, Zhize, et al.
出版: (2020)

Tutorial: "Advances in federated optimization: Efficiency, resiliency, and privacy"
由: CHI, Yuejie, et al.
出版: (2023)

CANITA: Faster rates for distributed convex optimization with communication compression
由: LI, Zhize, et al.
出版: (2021)

Acceleration for compressed gradient descent in distributed and federated optimization
由: LI, Zhize, et al.
出版: (2020)

CoVariance-based causal debiasing for entity and relation extraction
由: REN, Lin, et al.
出版: (2023)

Learning two-layer neural networks with symmetric inputs
由: GE, Rong, et al.
出版: (2019)

Gradient boosting with piece-wise linear regression trees
由: SHI, Yu, et al.
出版: (2019)

Optimal in-place suffix sorting
由: LI, Zhize, et al.
出版: (2018)

Conjugate gradient algorithms in nonconvex optimization
由: Pytlak, Radosław
出版: (2017)

SimpleNPKL: Simple Non-Parametric Kernel Learning
由: ZHUANG, Jinfeng, et al.
出版: (2009)

On top-k selection in multi-armed bandits and hidden bipartite graphs
由: CAO, Wei, et al.
出版: (2015)

SoteriaFL: A unified framework for private federated learning with communication compression
由: LI, Zhize, et al.
出版: (2022)

Simple effective named entity recognition for microblogs: Arabic as an example
由: DARWISH, Kareem, et al.
出版: (2014)

MARINA: Faster non-convex distributed learning with compression
由: GORBUNOV, Eduard, et al.
出版: (2021)

Coresets for vertical federated learning: Regularized linear regression and k-means clustering
由: HUANG, Lingxiao, et al.
出版: (2022)

Optimal in‐place suffix sorting
由: LI, Zhize, et al.
出版: (2022)

DeepACO: neural-enhanced ant systems for combinatorial optimization
由: YE, Haoran, et al.
出版: (2023)

3PC: Three point compressors for communication-efficient distributed training and a better theory for lazy aggregation
由: RICHTARIK, Peter, et al.
出版: (2022)

Neural multi-objective combinatorial optimization with diversity enhancement
由: CHEN, Jinbiao, et al.
出版: (2023)

Efficient meta neural heuristic for multi-objective combinatorial optimization
由: CHEN, Jinbiao, et al.
出版: (2023)

Hybrid stochastic-deterministic minibatch proximal gradient: Less-than-single-pass optimization with nearly optimal generalization
由: ZHOU, Pan, et al.
出版: (2020)

Dynamically-optimized context in recommender systems
由: YAP, Ghim-Eng, et al.
出版: (2005)

MetaBox: A benchmark platform for meta-black-box optimization with reinforcement learning
由: MA, Zeyuan, et al.
出版: (2023)

Evolutionary combinatorial optimization for recursive supervised learning with clustering
由: RAMANATHAN, Kiruthika, et al.
出版: (2007)

Chain of preference optimization: Improving chain-of-thought reasoning in LLMs
由: ZHANG, Xuan, et al.
出版: (2024)

Instance-level integration, query processing and optimization in Federated Database Systems
由: LIM, Ee Peng
出版: (1994)

STUDYING A SEQUENTIAL QUADRATIC PROGRAMMING ALGORITHM FOR NONCONVEX, NONSMOOTH CONSTRAINED OPTIMIZATION
由: SIOW YUN JIE
出版: (2021)

Personalized sentiment classification based on latent individuality of microblog users
由: SONG, Kaisong, et al.
出版: (2015)

Spatio‐temporal land use multi‐objective optimization: A case study in Central China
由: CAO, Kai, et al.
出版: (2020)

Monitoring the stability of the ALEPH vertex detector
由: SGUAZZONI, G., et al.
出版: (1999)

A near-optimal change-detection based algorithm for piecewise-stationary combinatorial semi-bandits
由: ZHOU, Huozhi, et al.
出版: (2020)