Enhancing in-vitro for comprehensive serverless cloud benchmarking

• Main Author: Kway, Yi Shen
• Other Authors: Dmitrii Ustiugov
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Workflows; Benchmarking; Serverless
• Online Access: https://hdl.handle.net/10356/181147
• Institution: Nanyang Technological University

In recent years, Function-as-a-Service (FaaS) systems have been rising in popularity among developers due to their cost-efficiency and ease of use. Many platforms, such as Microsoft Azure, AWS Lambda and Google Cloud offer a ”pay-as-you-go” model, where users are charged only for the duration their functions are running. Furthermore, workflow enhancements such as Azure Durable Functions and AWS Step functions seek to integrate stateful workflows for complex task execution. This provides considerable cost savings compared to traditional hosting models, where users incur infrastructure costs regardless of actual usage. However, as the inner workings of these systems are not available to the public, users would seek to compare the performance between them, leading to the rise of bench marking tools. While tools to benchmark performance in FaaS systems do exist, many of them focus on the commercial aspect, diving into cost optimisation, rather than research on performance and latency analysis. Those who do, have yet to implement analysis and benchmarks on workflows. In our project, we have successfully introduced workflows in our system, in the form of a Directed Acyclic Graph. We then measured the overhead of invocations and compared them by varying different aspects of the DAG. From our experiments, we derived several insights; DAG Length showed a linear relationship with latency, and DAG width did not have a clear impact on performance. Our project serves as an enhancement to an existing system that aims to synthesize representative workload summaries to evaluate the performance of serverless systems at diverse load scale factors