Federated learning study

• Main Author: Chan, Aloysius Zhen Wu
• Other Authors: Jun Zhao
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/148154
• Institution: Nanyang Technological University

Federated learning is a hot topic in the recent years due to the increased in emphasis for data privacy. Evidently, expert and specialised domain specific companies harbour large data assets required for a stronger machine learning model and these companies are generally not willing to disclose their restricted data to the public or other competing companies due to privacy infringement policies. Therefore, federated learning is one of the concepts introduced to allow training of machine learning models without direct knowledge of these companies’ restricted datasets, maintaining anonymity. However, as model architectures gets more robust and increasingly complex, model architectures will also correspondingly have an increase in its overall size. This causes longer model training durations which can take up a lot of time for the overall federated learning training process. Moreover, it also increases the computational requirements on the federated learning devices. This project will focus on constructing a federated learning framework to provide a proof of concept that with increasingly complex models, the overall federated learning process requires a much longer time for the training process. This is further emphasized by the lack of strong computational power on the targeted devices. Moreover, further experimentation such as the application of the state-of-the-art deep reinforcement learning (DRL) pruning can be applied to show the effects of model pruning on the whole federated learning process. The demonstration is done with multiple python scripts to simulate the federated learning framework. Multiple raspberry PI devices will also be used to simulate companies hosting their respective training datasets. For the additional experimentation of DRL pruning, the pruning ratio can be adjusted on the pruning script to change the model architectures to specific targeted sizes.