Deep learning for UI testing

• Main Author: Koh, Hong Da
• Other Authors: Liu Yang
• Format: Final Year Project
• Language: English
• Published: 2018
• Subjects: DRNTU::Science::Medicine::Computer applications
• Online Access: http://hdl.handle.net/10356/74227
• Institution: Nanyang Technological University

User Interface (UI) Testing of Android Applications has been greatly overlooked by many developers in the recent years despite proliferation in application usage. One of the reason is because of the inefficiency in undergoing UI tests to search for crashes despite the many benefits of doing so. This report seeks to improve the efficiency of UI testing while ensuring coverage remains high. To achieve this, we collect data from the many applications on Google Play, use them to train a deep learning model before determining the accuracy and correlation between underlying UI hierarchy data with exploration rates. We document various experiments done with four different training models: fastText from Facebook, Logistic Regression Model, Recurrent Neural Network (RNN) using the Long Short-Term Memory (LSTM) model as well as the Wide and Deep Model with the latter three using Tensorflow. We also document the design and implementation of the automated tool which is used in collecting data as well as the relevant data collected. Our results suggest an optimal accuracy of 85% when using both the Recurrent Neural Network Model and Wide and Deep Model. As for fast- Text and the Logistic Regression Model, our results show a poor correlation primarily because of the differences in data used.