Efficient Virtualization-based Application Protection against Untrusted Operating System

• Main Authors: CHENG, Yueqiang, DING, Xuhua, DENG, Robert H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2015
• Subjects: application protection; isolated execution environment; address space isolation; untrusted OS; Computer Sciences; Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/2880
• Institution: Singapore Management University

Commodity monolithic operating systems are abundant with vulnerabilities that lead to rootkit attacks. Once an operating system is subverted, the data and execution of user applications are fully exposed to the adversary, regardless whether they are designed and implemented with security considerations. Existing application protection schemes have various drawbacks, such as high performance overhead, large Trusted Computing Base (TCB), or hardware modification. In this paper, we present the design and implementation of AppShield, a hypervisor-based approach that reliably safeguards code, data and execution integrity of a critical application, in a more efficient way than existing systems. The protection overhead is localized to the protected application only, so that unprotected applications and the operating system run without any performance loss. In addition to the performance advantage, AppShield tackles several newly identified threats in this paper which are not systematically addressed previously. We build a prototype of AppShield with a tiny hypervisor, and experiment with AppShield by running several off-the-shelf applications on a Linux platform. The results testify to AppShield's low performance costs in terms of CPU computation, disk I/O and network I/O.