RopSteg: Program steganography with return oriented programming

Many software obfuscation techniques have been proposed to hide program instructions or logic and to make reverse engineering hard. In this paper, we introduce a new property in software obfuscation, namely program steganography, where certain instructions are "diffused" in others in such...

Full description

Saved in:
Bibliographic Details
Main Authors: LU, Kangjie, XIONG, Siyang, GAO, Debin
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2014
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7033
https://ink.library.smu.edu.sg/context/sis_research/article/8036/viewcontent/RopSteg_pv.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:Many software obfuscation techniques have been proposed to hide program instructions or logic and to make reverse engineering hard. In this paper, we introduce a new property in software obfuscation, namely program steganography, where certain instructions are "diffused" in others in such a way that they are non-existent until program execution. Program steganography does not raise suspicion in program analysis, and conforms to the W⊕X and mandatory code signing security mechanisms. We further implement RopSteg, a novel software obfuscation system, to provide (to a certain degree) program steganography using return-oriented programming. We apply RopSteg to eight Windows executables and evaluate the program steganography property in the corresponding obfuscated programs. Results show that RopSteg achieves program steganography with a small overhead in program size and execution time. RopSteg is the first attempt of driving return-oriented programming from the "dark side", i.e., using return-oriented programming in a non-attack application. We further discuss limitations of RopSteg in achieving program steganography.