Vulnerability detection on web browsers

• Main Author: Yu, Haiwan
• Other Authors: WU Hongjun
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Software::Software engineering
• Online Access: https://hdl.handle.net/10356/143063
• Institution: Nanyang Technological University

Web browser is the most commonly used software to access the Internet. Any vulnerability in a popular web browser will compromise the security and privacy of its users. With the increasing complexity of modern web browsers, the attack surface of web browser increased dramatically and more vulnerabilities were introduced. In this thesis, we developed a fuzzing framework to detect vulnerability in the web browser. Our fuzzing framework is designed for large scale fuzzing of all the popular web browsers running on virtual machines. Our fuzzing framework supports fuzzing with multiple test case generation strategies in a test case generator set, and test case generation strategies can be changed when fuzzer is running. By running this fuzzing framework together with our various vulnerability detection methods, many crashes were found and in total 5 CVEs were assigned to the vulnerabilities being found. In this thesis, we proposed a new type of vulnerability, namely, the memory pressure bugs. This type of vulnerability was triggered by failed memory allocation. Using the existing fuzzing methods, it is extremely hard to trigger this type of bugs. It is also extremely difficult to reproduce this type of bugs since reproducing the crashes requires the identical memory allocation to be failed, while the memory allocations in computer system are hard to predict in general. To trigger this type of bugs, we developed low memory simulation instrumentation tools to aid our fuzzer to detect memory pressure bugs in web browsers. To reproduce this type of bugs, we introduced precise memory pressure in JavaScript code. We solve the problem of premature allocation failure of memory pressure bug by leveraging on the memory fragmentation to reserve memory space for allocation before the target allocation. Three new vulnerabilities of memory pressure bugs were successfully found in the Internet explorer. In this Thesis, we analyzed 5 CVEs we have found and a zero-day vulnerability in Internet explorer. We exclusively disclose the details of these 6 vulnerabilities and proof of concept (POC) to trigger them.