ROPSentry : runtime defense against ROP attacks using hardware performance counters

• Main Authors: Das, Sanjeev, Chen, Bihuan, Chandramohan, Mahintham, Liu, Yang, Zhang, Wei
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2020
• Subjects: Engineering::Electrical and electronic engineering; ROP Attacks; Hardware Performance Counter
• Online Access: https://hdl.handle.net/10356/142012
• Institution: Nanyang Technological University

Return-Oriented Programming (ROP) is one of the most common techniques to exploit software vulnerabilities. However, existing defense techniques can be defeated by attackers, or suffer from high performance overhead. In this paper, we propose a defense framework, named ROPSentry, to detect ROP attacks at runtime. It is built on the observation that ROP exploits usually trigger different hardware events than normal programs generated by compilers. Hence, we leverage hardware performance counters to track such hardware events and analyze behavioral patterns of ROP attacks. ROPSentry has two approaches. The ROP-only defense approach detects ROP attacks via capturing the patterns of ROP exploits, where we propose to sample the hardware performance counters at mispredicted return events instead of at every microinstruction for a low performance overhead. To further reduce performance overhead, we propose a self-adaptive defense approach to dynamically switch between low and high sampling rates. It detects the patterns of spraying attacks (i.e., one common ROP payload delivery technique) at a low sampling rate, and then switches to a high sampling rate for detecting the patterns of ROP exploits. Our evaluation on 11 real-world ROP exploits, 50 synthetically generated ROP exploits and 1000 benign websites has shown that, the ROP-only and self-adaptive approaches are effective in detecting ROP attacks with low performance overhead (11% and 1% respectively) as well as low false positive; and they significantly outperform the state-of-the-art techniques in terms of performance overhead without losing the detection accuracy.