Towards distributed machine learning in shared clusters: A dynamically-partitioned approach

• Main Authors: SUN, Peng, WEN, Yonggang, TA, Nguyen Binh Duong, YAN, Shengen
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Artificial Intelligence and Robotics; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/4766; https://ink.library.smu.edu.sg/context/sis_research/article/5769/viewcontent/1704.06738.pdf
• Institution: Singapore Management University

Many cluster management systems (CMSs) have been proposed to share a single cluster with multiple distributed computing systems. However, none of the existing approaches can handle distributed machine learning (ML) workloads given the following criteria: high resource utilization, fair resource allo...