An analytic layer-wise learning framework for deep networks with applications to robotics

• Main Author: Nguyen, Huu Thiet
• Other Authors: Cheah Chien Chern
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/159034
• Institution: Nanyang Technological University

With increasing involvement of robots in human activities, robot applications are no longer limited to manipulators performing repetitive tasks in isolated environments. Robot systems nowadays can be found in diversified forms doing increasingly complex tasks in changing environments. This makes the modeling and control tasks in robotics more difficult. Employing artificial neural networks has been a useful approach for approximating the uncertainties in robot control. Since stability is crucial for safe robot operations, most existing works focuses on utilizing shallow networks where the learning is applied mostly for the output weights so as to simplify the analysis to ensure convergence. However, these shallow network approaches may not demonstrate the full potential of using deeper networks in robotic applications. Deep networks are generally better than shallow counterparts when it comes to the approximation and generalization capabilities. In fact, in machine learning community, many deep models which are trained by deep learning algorithms have been achieving tremendous results in numerous classification applications. The reason behind the success of deep learning, however, remains unclear as deep learning models are usually considered as black boxes. Deep learning has been less well analyzed from the theoretical perspective and its convergence issue has not received sufficient considerations. This may pose risks when deep learning is applied in robot control because robots are active agents which interact physically with the outside world. Failing to converge in robot control may cause direct accidents that are not usually seen in machine learning applications such as classifications. In this thesis, an analytic layer-wise deep learning framework is developed in which the convergence can be analyzed from theoretical perspective. Using the proposed framework, deep networks which include fully connected networks and convolutional neural networks are built and trained progressively in a layer-wise manner such that the convergence of the learning systems is ensured. Unlike existing works, the proposed methodology can be used safely in different robotic applications centered at controlling robots. It is compatible with both regression and classification problems and can be employed for both offline and online learning in robot control tasks. For offline classification problems, it is drawn from different benchmark case studies using common machine learning datasets that the accuracies of the proposed framework are comparable to those of state-of-the-art gradient descent method. As for regression problems in online robot control tasks, experimental results are presented to illustrate the advantages of the proposed approach over existing shallow network approaches as well as gradient descent method.