Serverless cloud benchmarking with serverless function traces

• Main Author: Lee, Xuan Hua
• Other Authors: Dmitrii Ustiugov
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science
• Online Access: https://hdl.handle.net/10356/175344
• Institution: Nanyang Technological University

With the emergence of cloud computing and popularity of Function-as-a-Service (FaaS) systems, many cloud providers have offered serverless solutions, masking the traditional server complexities. As the implementation of their scheduler and system is not made known to developers, this black-box nature has resulted in the widening information gaps between providers and application developers, who share different Quality of Service goals. This raises questions about performance comparisons across providers. While prior works have been made to benchmark performance across providers, most focused on creating representative workloads for specific measurements on the latency and performance of typical applications. Many do not explore the sources of performance degradations. Even when they do, the research are primarily on avoidance of cold start invocations or are limited to specific factors such as language runtime. In our project, we identify factors significant to the system performance – in minimising the number of cold start invocations, and more importantly, in reducing the cold start latencies. Through our experimentation, we found evidence of resource sharing across different functions and within the same function. Factors discovered to influence cold start latencies include: the stagger duration between functions, the number of deployed functions, and the presence of 1 warm instance (non-zero scaling). Factors found to not affect cold start latencies include: the function execution duration, and the size of the invocation step. Lastly, we explored the fixed keep alive policy of AWS to derive insights to minimise the number of cold start invocations. Finally, our project is part of a wider effort to build a system capable of analysing existing serverless solutions with provider-agnostic methodology, receiving inputs from both real-world workloads and a set of purposely built synthetic workloads.