Probabilistic path prioritization for hybrid fuzzing

• Main Authors: ZHAO, Lei, CAO, Pengcheng, DUAN, Yue, YIN, Heng, XUAN, Jifeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Software security; Fuzz testing; Concolic execution; Hybrid fuzzing; Information Security; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/8198; https://ink.library.smu.edu.sg/context/sis_research/article/9201/viewcontent/ProbPathPrioritization_2022_av.pdf
• Institution: Singapore Management University

Hybrid fuzzing that combines fuzzing and concolic execution has become an advanced technique for software vulnerability detection. Based on the observation that fuzzing and concolic execution are complementary in nature, state-of-the-art hybrid fuzzing systems deploy “optimal concolic testing” and “demand launch” strategies. Although these ideas sound intriguing, we point out several fundamental limitations in them, due to unrealistic or oversimplified assumptions. Further, we propose a novel “discriminative dispatch” strategy and design a probabilistic hybrid fuzzing system to better utilize the capability of concolic execution. Specifically, we design a Monte Carlo-based probabilistic path prioritization model to quantify each path’s difficulty, and then prioritize them for concolic execution. Our model assigns the most difficult paths to concolic execution. We implement a prototype named DigFuzz and evaluate our system with two representative datasets and real-world programs. Results show that the concolic execution in DigFuzz outperforms than those in state-of-the-art hybrid fuzzing systems in every major aspect. In particular, the concolic execution in DigFuzz contributes to discovering more vulnerabilities (12 versus 5) and producing more code coverage (18.9 versus 3.8 percent) on the CQE dataset than the concolic execution in Driller.