Acceleration for compressed gradient descent in distributed and federated optimization

• Main Authors: LI, Zhize, KOVALEV, Dmitry, QIAN, Xun, RICHTARIK, Peter
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8681; https://ink.library.smu.edu.sg/context/sis_research/article/9684/viewcontent/ICML20_full_adiana.pdf
• Institution: Singapore Management University

Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While in other contexts the best performing gradient-type methods invariably rely on some form of acceleration/momentum to red...