Behaviour-based malware detection on Android phones

• Main Author: Lim, Jing Qiang
• Other Authors: Liu Yang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computing methodologies
• Online Access: https://hdl.handle.net/10356/144502
• Institution: Nanyang Technological University

With the proliferation of Android malware, the demand for an effective and efficient malware detection system is on the rise. The existing device-end learning based solutions tend to extract limited syntax features, such as permissions and API calls, to meet a certain time constraint of mobile devices. However, unlike sequence-based feature, syntax feature lacks the semantics which can represent the potential malicious behaviors and further results in more robust model with high accuracy for malware detection. This report introduces an efficient Android malware detection system, named SeqMobile, which adopts behavior-based sequence features and leverages customized deep neural networks on mobile devices instead of the server end. Different from the traditional sequence-based approaches on server end, to meet the performance demand on mobile devices, SeqMobile accepts three effective performance optimization methods to reduce the time of feature extraction and prediction. To evaluate the effectiveness and efficiency of SeqMobile, we conduct experiments from the following aspects 1) the detection accuracy of different recurrent neural networks (RNN); 2) the feature extraction performance on different mobile devices, and 3) the detection accuracy and prediction time cost of different sequence length. The results unveil that SeqMobile can effectively detect malware with high accuracy. Moreover, the proposed performance optimization methods have proven to improve the performance of training and prediction time by at least twofold. Additionally, to discover the potential performance optimization from the state-of-the-art TensorFlow model optimization toolkit for sequence-based approaches, an evaluation was conducted on the toolkit, which can serve as a guidance for other systems leveraging on sequence-based learning approach. Overall, using a sequence-based approach, together with the proposed performance optimization methods, enable SeqMobile to efficiently detect malware under the performance demands of mobile devices.