Development of a learning algorithm for convolutional neural networks

• Main Author: Fu, Jiadi
• Other Authors: Cheah Chien Chern
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/164462
• Institution: Nanyang Technological University

Over the years, deep learning has become one of the most popular topics in computer science. By training artificial neural networks on large datasets, deep learning algorithms can learn to recognize patterns and features in data and use this learning to make intelligent decisions. That is why it has been widely used in linguistics and image processing. There are different training methods faced to various scenarios, while layer-wise training is one of the typical ways. Layer-wise training is a method in which the layers of the model are trained one at each time, rather than all at once. Reduced complexity helps layer-wise training be more efficient than training the entire model at once. However, it brings potential issues like suboptimal models and overfitting. Layer-wise training can be effective if only applied to appropriate scenarios and methods. This report aimed at exploring the possibility of improving the performance of layer-wise learning by using adjustable ReLu and autoencoder structures. To prove the ability of this training method, two groups of models are constructed based on VGG-11 and Autoencoder, applying in classification and image reconstruction tasks separately. The results related to these experiments are proposed to prove their feasibility. The optimized models improve the learning efficiency and convergence speed. Apart from achieving similar accuracy to the original end-to-end models, layer-wise training improves the convergence speed of the model and reduces the gradient calculation between layers, which has been proved to be a more effective method.