Linear Obfuscation to Combat Symbolic Execution

• Main Authors: WANG, Zhi, Ming, Jiang, Jia, Chunfu, GAO, Debin
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2011
• Subjects: Software obfuscation; symbolic execution; malware analysis; Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/2005; https://ink.library.smu.edu.sg/context/sis_research/article/3004/viewcontent/esorics11.pdf
• Institution: Singapore Management University

Trigger-based code (malicious in many cases, but not necessarily) only executes when specific inputs are received. Symbolic execution has been one of the most powerful techniques in discovering such malicious code and analyzing the trigger condition. We propose a novel automatic malware obfuscation technique to make analysis based on symbolic execution difficult. Unlike previously proposed techniques, the obfuscated code from our tool does not use any cryptographic operations and makes use of only linear operations which symbolic execution is believed to be good in analyzing. The obfuscated code incorporates unsolved conjectures and adds a simple loop to the original code, making it less than one hundred bytes longer and hard to be differentiated from normal programs. Evaluation shows that applying symbolic execution to the obfuscated code is inefficient in finding the trigger condition. We discuss strengths and weaknesses of the proposed technique.