Software protection system : binary control flow obfuscation for portable executables

• Main Author: Chew, Ron Jun Hong.
• Other Authors: Sabu Emmanuel
• Format: Final Year Project
• Language: English
• Published: 2012
• Subjects: DRNTU::Engineering::Computer science and engineering::Software::Software engineering; DRNTU::Engineering::Computer science and engineering::Data::Data encryption
• Online Access: http://hdl.handle.net/10356/48503
• Institution: Nanyang Technological University

With the advancement of reverse engineering technology, the rise in software piracy and program hacking is becoming a worrying factor for software developers. Software developers are always on the lookout for cost-effective measures to protect their software. Even though there are no definite guarantees in software protection, it is possible to devise a complex obfuscation technique to make program reverse-engineering time-consuming and impracticable. Binary code obfuscation demonstrates the ability to obscure program structure, content and behavior while retaining program functionality. This project features a custom devised obfuscation design that incorporates several modified binary-level obfuscation techniques. These obfuscation techniques include control flow flattening and branch conversion and they are directed at the control flow of a program to hide its intended logic and behavior. The proposed obfuscation design is then tested and implemented into a working prototype that serves as an automated obfuscator for windows portable executables (PE). Unlike other obfuscation techniques implemented at source-code level, the obfuscator performs direct binary code modifications to the targeted PE program. Obfuscated programs are then evaluated against several performance metrics to determine the efficiency of the proposed obfuscation. Experimental results indicate high confusion potency and resilience in the obfuscation techniques. These factors represent how well the obfuscation can confuse and thwart disassembly.