Efficient gradient support pursuit with less hard thresholding for cardinality-constrained learning

• Main Authors: SHANG, Fanhua, WEI, Bingkun, LIU, Hongying, LIU, Yuanyuan, ZHOU, Pan, GONG, Maoguo
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Complexity theory; Stochastic processes; Convergence; Indexes; Estimation; Approximation algorithms; Sparse matrices; Hard thresholding (HT); sparse learning; sparsity; rank-constrained problem; stochastic optimization; variance reduction; OS and Networks
• Online Access: https://ink.library.smu.edu.sg/sis_research/9049
• Institution: Singapore Management University

Recently, stochastic hard thresholding (HT) optimization methods [e.g., stochastic variance reduced gradient hard thresholding (SVRGHT)] are becoming more attractive for solving large-scale sparsity/rank-constrained problems. However, they have much higher HT oracle complexities, especially for high...