Partitioning uncertain workloads

• Main Authors: CHUA, Freddy, HUBERMAN, Bernardo A.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2016
• Subjects: Parallelization; Partitioning; Workflow; Uncertainty; Optimization; Machines; Computer Sciences; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/3974; https://ink.library.smu.edu.sg/context/sis_research/article/4976/viewcontent/Partitioning_uncertain_workloads_2016_afv.pdf
• Institution: Singapore Management University

We present a method for determining the ratio of the tasks when breaking any complex workload in such a way that once the outputs from all tasks are joined, their full completion takes less time and exhibit smaller variance than when running on the undivided workload. To do that, we have to infer the capabilities of the processing unit executing the divided workloads or tasks. We propose a Bayesian Inference algorithm to infer the amount of time each task takes in a way that does not require prior knowledge on the processing unit capability. We demonstrate the effectiveness of this method in two different scenarios; the optimization of a convex function and the transmission of a large computer file over the Internet. Then we show that the Bayesian inference algorithm correctly estimates the amount of time each task takes when executed in one of the processing units.