RopSteg: Program steganography with return oriented programming

• Main Authors: LU, Kangjie, XIONG, Siyang, GAO, Debin
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2014
• Subjects: code obfuscation; program steganography; return-oriented programming; watermarking; Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/7033; https://ink.library.smu.edu.sg/context/sis_research/article/8036/viewcontent/RopSteg_pv.pdf
• Institution: Singapore Management University

Many software obfuscation techniques have been proposed to hide program instructions or logic and to make reverse engineering hard. In this paper, we introduce a new property in software obfuscation, namely program steganography, where certain instructions are "diffused" in others in such a way that they are non-existent until program execution. Program steganography does not raise suspicion in program analysis, and conforms to the W⊕X and mandatory code signing security mechanisms. We further implement RopSteg, a novel software obfuscation system, to provide (to a certain degree) program steganography using return-oriented programming. We apply RopSteg to eight Windows executables and evaluate the program steganography property in the corresponding obfuscated programs. Results show that RopSteg achieves program steganography with a small overhead in program size and execution time. RopSteg is the first attempt of driving return-oriented programming from the "dark side", i.e., using return-oriented programming in a non-attack application. We further discuss limitations of RopSteg in achieving program steganography.