To detect stack buffer overflow with polymorphic canaries

• Main Authors: WANG, Zhilong, DING, Xuhua, PANG, Chengbin, GUO, Jian, ZHU, Jun, MAO, Bing
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Brute force attack; Canary; Stack buffer overflow; Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/4101; https://ink.library.smu.edu.sg/context/sis_research/article/5104/viewcontent/Stack_Buffer_Polymorphic_Canaries_2018_DSN_afv.pdf
• Institution: Singapore Management University

Stack Smashing Protection (SSP) is a simple and highly efficient technique widely used in practice as the front line defense against stack buffer overflow attacks. Unfortunately, SSP is known to be vulnerable to the so-called byte-by-byte attack. Although several remedy schemes are proposed in the recent literature, their security is achieved at the price of practicality, because their complex logics ruin SSP's simplicity and high-efficiency. In this paper, we present an elegant solution named as Polymorphic SSP (P-SSP) that attains the same security without sacrificing SSP's strengths. We also propose three extensions of the basic scheme for better compatibility, stronger security, and local variable protection, respectively. We have implemented both a compiler plugin and a binary instrumentation tool for deploying P-SSP. Their respective runtime overheads are only 0.24% and 1.01%. We have also experimented with our extensions and compared their pros and cons with the basic scheme.