Privacy-preserving deep learning

• Main Authors: Yik, Jia Ler, Tan, Zhen Yuan, Zaw, Maw Htun
• Other Authors: Lam Kwok Yan
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/151686
• Institution: Nanyang Technological University

Data is coined to be the new oil due to the increasing awareness of its value in a myriad of applications running the gamut from automating personalised services to artificial intelligence - all of which with machine learning (ML) at their core. With this rising trend, there is growing attention on privacy by consumers and government bodies; this introduces the need for Federated Learning (FL) and Differential Privacy (DP) - an evolved form of ML, where models are trained while privacy is safeguarded - which forms the focus of our research. We visited existing research developments in privacy-preserving deep learning applications on structured and unstructured data and designed a proof-of-concept platform for the same, in the form of a Convolutional Neural Network for MNIST dataset handwritten digits hosted on the Cloud. Our experiment structure tested the different permutations between the degree of training in models, determined by the number of epochs per generation, and whether DP was implemented. In particular, our findings indicated the following: 1, adding noise to trained weights resulted in an overall decrease in trained accuracy but greater epsilon value; 2, larger locally trained accuracy for a larger epoch run presented itself with a larger accuracy drop; 3, lower final validation accuracy was achieved for DP models; 4, there was a low correlation between final validation accuracy values with standard deviation regardless of DP model. Further research can be conducted on the differing FL structures and centrality. Although FL is relatively new, there is strong evidence to suggest a growing interest and attention towards it. We hold the opinion that FL has a place in collaborative ML-based applications while preserving the privacy of end-users.