Topology-aware hypergraph based approach to optimize scheduling of parallel applications onto distributed parallel architectures

• Main Author: Koohi, Sina Zangbari
• Format: Thesis
• Language: English
• Published: 2020
• Subjects: Parallel programming (Computer science) - Research; Parallel processing (Electronic computers); Hypergraphs
• Online Access: http://psasir.upm.edu.my/id/eprint/99242/1/SINA%20ZANGBARI%20KOOHI%20IR.pdf; http://psasir.upm.edu.my/id/eprint/99242/
• Institution: Universiti Putra Malaysia

It has broadly acknowledged that the rapid progression of computer technology has brought dramatic growth in the complexity and scale of systems. These complex systems have designed to solve various types of problems from different areas, resulting in high-demanding Heterogeneous Parallel Applications (HPAs). HPAs use parallel processors and assist in parallel execution of tasks with complex interdependency between data and operations. In such architectures, having less waiting time and less response time are crucial. However, achieving an optimum solution for these two metrics is a trivial task because their efficiency relies on modelling, optimizing, partitioning, and job scheduling methods. In this thesis, an approach to optimize the scheduling of parallel applications over heterogeneous architectures to achieve optimum waiting and response time has proposed. The proposed technique has provided through four fundamental steps, including modelling of parallel applications, meta-heuristic optimization, partitioning, and parallel job scheduling. The first step lies at the modelling of parallel applications running on heterogeneous parallel computers. Modelling refers to constructing a model to depict the structure of the application with its tasks and describing the interactions between them. The existing modelling approaches capture the processor heterogeneity information in the model. However, the network heterogeneity has not considered xv before, and the crucial data to reflect the network heterogeneity are missing. Consequently, the metrics provided by them does not cover the network heterogeneity. The first contribution of this thesis is to propose a new modelling approach named MEMPHA that would consider heterogeneity and capture all vital metrics, resulting in more accurate modelling of HPAs. MEMPHA, a hypergraphs-based model, aims to aspire to the challenge by providing topology modelling of the target parallel machine and application modelling of the parallel application, which is hypergraph-based model, to abstract the details of HPAs. To demonstrate the effectiveness of MEMPHA, experiments have performed on a set of benchmark hypergraphs. As a result, when compared with previous modelling approaches, MEMPHA shows promising results in devising a better plan for assignments of tasks to processors, which in turn aims to achieve better performance. Since scheduling and mapping fall into NP problems, and there is no efficient exact solution for solving scheduling and mapping, the second challenge in HPAs is optimization. Meta-heuristic algorithms have widely used in HPAs due to their global optimization ability. However, the current meta-heuristic algorithms do not ensure an optimum solution within a reasonable time. Hence, there is yet room for improvement. Moreover, evolutionary algorithms are generally limited in their problemsolving abilities. Any optimization algorithm is suitable for only a specific domain of optimization problems. For these reasons, to improve the time and accuracy of the coverage in population-based meta-heuristics and their utilization in HPAs, this thesis presents a novel optimization algorithm called the Raccoon Optimization Algorithm (ROA). Mimicking a raccoon’s search behaviour, the ROA concentrates its searches in the solution space of non-linear continuous problems at finding the global optimum with higher accuracy and lower time coverage. To evaluate the capability of ROA at addressing complicated problems, it has subjected to experiment several benchmark functions. The ROA has then compared with nine well-known optimization algorithms. Subsequent results show that the ROA performs at a higher accuracy with lower coverage time. The core approach in task scheduling is the partitioning of the tasks, which devise their distribution pattern over processors. Tasks partitioning refers to the effort of grouping tasks into several sets. Providing a balanced partitioning with equal weights are widely studied. However, in heterogeneous architecture, process heterogeneity demands partitions with different weights. Thus, an efficient partitioning to find an optimum dividing of a hypergraph into K imbalance partitioning is the third challenge of HPAs. This thesis provides a new topology-aware multi-level hypergraph partitioning schema to tackle this issue. The proposed partitioning scheme has based on a multi-level partitioning approach which consists of three main steps. In the first step, a sequence of coarsening on the hypergraph has applied to achieve a smaller coarsened hypergraph. Then, in the second phase, the coarsened hypergraph is partitioned to obtain the initial partitions. Finally, the initial partitioning is successively un-coarsened and re-refined back to the original hypergraph. These steps have conducted using the MEMPHA model and ROA algorithm to optimize three metrics: execution time, total communication volume, and imbalance ratio (load balancing). To experiment the efficiency of the proposed topology-aware multi-level hypergraph partitioning schema, a set of benchmark hypergraphs have used. The results have compared with other multi-level hypergraph partitioning tools and indicates that the proposed approach achieve optimum partitioning and significantly increase the speed of partitioning. The final step in scheduling and mapping is the distribution of the jobs. Job distribution (Job scheduling) refers to planning the order and layout of execution for all submitted jobs. An inefficient layout yields to higher waiting and low speed response time. To achieve an optimum waiting and response time this thesis has proposed a new approach utilizing the aforementioned modelling, optimizing and partitioning algorithms. This approach has simulated on Alea v.4, which is a dedicated simulator for simulating exascale parallel scheduling. The results have compared with multiple scheduling methods and indicated that the proposed method achieves substantial performance improvements in terms of reducing the average waiting time and response time of the jobs.