Mars-HD : a high performance computing systems for many-core processors

• Main Author: Lim, Chun Leng
• Other Authors: School of Computer Engineering
• Format: Final Year Project
• Language: English
• Published: 2014
• Subjects: DRNTU::Engineering::Computer science and engineering::Computer systems organization::Performance of systems
• Online Access: http://hdl.handle.net/10356/59263
• Institution: Nanyang Technological University

Mars-HD was built over Mars, a GPU MapReduce Framework, to allow it to work in the Hadoop cluster. This gives rise to the ability for multiple GPUs to work together in a distributed cloud system. As data processing becomes more crucial to understanding the trends of the world, it becomes essential to process these incoming data in the shortest time possible. Thus, the power of the clusters is necessary. Mars-HD successfully ensure scalability of the Mars framework, by building it directly on top of Hadoop MapReduce Framework, allowing Hadoop to perform the function of load balancing and then sends the data over to Mars to be processed and outputs the result. Performance on Mars-HD is not on par with Mars due to the overhead in the Hadoop Framework, however, with the use of Mars-HD on an input data of the correct size, the performance difference can be reduced and achieve speedup.