Locating vulnerabilities in binaries via memory layout recovering

• Main Authors: WANG, Haijun, XIE, Xiaofei, LIN, Shang-Wei, LIN, Yun, LI, Yuekang, QIN, Shengchao, LIU, Yang, LIU, Ting
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Reverse Engineering; Software Vulnerability; Program Analysis; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7066; https://ink.library.smu.edu.sg/context/sis_research/article/8069/viewcontent/3338906.3338966.pdf
• Institution: Singapore Management University

Locating vulnerabilities is an important task for security auditing, exploit writing, and code hardening. However, it is challenging to locate vulnerabilities in binary code, because most program semantics (e.g., boundaries of an array) is missing after compilation. Without program semantics, it is difficult to determine whether a memory access exceeds its valid boundaries in binary code. In this work, we propose an approach to locate vulnerabilities based on memory layout recovery. First, we collect a set of passed executions and one failed execution. Then, for passed and failed executions, we restore their program semantics by recovering fine-grained memory layouts based on the memory addressing model. With the memory layouts recovered in passed executions as reference, we can locate vulnerabilities in failed execution by memory layout identification and comparison. Our experiments show that the proposed approach is effective to locate vulnerabilities on 24 out of 25 DARPA’s CGC programs (96%), and can effectively classifies 453 program crashes (in 5 Linux programs) into 19 groups based on their root causes.