Layer-wise learning framework for deep networks

• Main Author: Yu, Haoyao
• Other Authors: Cheah Chien Chern
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Layer-wise learning
• Online Access: https://hdl.handle.net/10356/179107
• Institution: Nanyang Technological University

With the increasingly extensive application of deep learning, research based on experiential results has made significant progress in the field of machine learning over the past few years. However, deep learning is very difficult to understand due to its use of artificial neural networks and a black-box approach. A lack of knowledge about deep learning networks will hinder their development in situations where taking large risks is necessary, as well as restrict their use in situations where robust, dependable artificial intelligence is desired. The aim of this dissertation is to use the stochastic gradient descent method, but hierarchically, to train deep residual neural networks for better understanding. Firstly, it is of great importance to construct a mathematical model for deep residual neural networks based on matrix forms and then hierarchically train and test the deep residual neural networks on a few common image datasets employing the stochastic gradient descent technique. The case examples demonstrate a rational compromise regarding layer-wise trainability and precision while validating the applicability of the proposed layer-wise learning method to determine the optimal number of layers for real-world scenarios.