Detection and classification of malicious JavaScript via attack behavior modelling

Existing malicious JavaScript (JS) detection tools and commercial anti-virus tools mostly use feature-based or signature-based approaches to detect JS malware. These tools are weak in resistance to obfuscation and JS malware variants, not mentioning about providing detailed information of attack beh...

Full description

Saved in:
Bibliographic Details
Main Authors: XUE, Yinxing, WANG, Junjie, LIU, Yang, XIAO, Hao, SUN, Jun, CHANDRAMOHAN, Mahinthan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2015
Subjects:
L*
Online Access:https://ink.library.smu.edu.sg/sis_research/4953
https://ink.library.smu.edu.sg/context/sis_research/article/5956/viewcontent/issta2015.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:Existing malicious JavaScript (JS) detection tools and commercial anti-virus tools mostly use feature-based or signature-based approaches to detect JS malware. These tools are weak in resistance to obfuscation and JS malware variants, not mentioning about providing detailed information of attack behaviors. Such limitations root in the incapability of capturing attack behaviors in these approches. In this paper, we propose to use Deterministic Finite Automaton (DFA) to abstract and summarize common behaviors of malicious JS of the same attack type. We propose an automatic behavior learning framework, named JS∗ , to learn DFA from dynamic execution traces of JS malware, where we implement an effective online teacher by combining data dependency analysis, defense rules and trace replay mechanism. We evaluate JS∗ using real world data of 10000 benign and 276 malicious JS samples to cover 8 most-infectious attack types. The results demonstrate the scalability and effectiveness of our approach in the malware detection and classification, compared with commercial anti-virus tools. We also show how to use our DFAs to detect variants and new attacks.