A layer-wise theoretical framework for deep learning of convolutional neural networks
As research attention in deep learning has been focusing on pushing empirical results to a higher peak, remarkable progress has been made in the performance race of machine learning applications in the past years. Yet deep learning based on artificial neural networks still remains difficult to under...
Saved in:
Main Authors: | , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2022
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/155225 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-155225 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1552252022-02-14T00:40:09Z A layer-wise theoretical framework for deep learning of convolutional neural networks Nguyen, Huu-Thiet Li, Sitan Cheah, Chien Chern School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Deep learning Trust in AI Robot Learning Robot Vision As research attention in deep learning has been focusing on pushing empirical results to a higher peak, remarkable progress has been made in the performance race of machine learning applications in the past years. Yet deep learning based on artificial neural networks still remains difficult to understand as it is considered as a black-box approach. A lack of understanding of deep learning networks from the theoretical perspective would not only hinder the employment of them in applications where high-stakes decisions need to be made, but also limit their future development where artificial intelligence is expected to be robust, predictable and trustable. This paper aims to provide a theoretical methodology to investigate and train deep convolutional neural networks so as to ensure convergence. A mathematical model based on matrix representations for convolutional neural networks is first formulated and an analytic layer-wise learning framework for convolutional neural networks is then proposed and tested on several common benchmarking image datasets. The case studies show a reasonable trade-off between accuracy and analytic learning, and also highlight the potential of employing the proposed layer-wise learning method in finding the appropriate number of layers in actual implementations. Agency for Science, Technology and Research (A*STAR) Published version This work was supported by the Agency for Science, Technology and Research of Singapore (A*STAR), under the National Robotics Program—Robotics Domain Specific, under Grant 1922200001. 2022-02-14T00:40:09Z 2022-02-14T00:40:09Z 2022 Journal Article Nguyen, H., Li, S. & Cheah, C. C. (2022). A layer-wise theoretical framework for deep learning of convolutional neural networks. IEEE Access, 10, 14270-14287. https://dx.doi.org/10.1109/ACCESS.2022.3147869 2169-3536 https://hdl.handle.net/10356/155225 10.1109/ACCESS.2022.3147869 10 14270 14287 en 1922200001 IEEE Access © 2022 IEEE. This journal is 100% open access, which means that all content is freely available without charge to users or their institutions. All articles accepted after 12 June 2019 are published under a CC BY 4.0 license, and the author retains copyright. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, as long as proper attribution is given. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Electrical and electronic engineering Deep learning Trust in AI Robot Learning Robot Vision |
spellingShingle |
Engineering::Electrical and electronic engineering Deep learning Trust in AI Robot Learning Robot Vision Nguyen, Huu-Thiet Li, Sitan Cheah, Chien Chern A layer-wise theoretical framework for deep learning of convolutional neural networks |
description |
As research attention in deep learning has been focusing on pushing empirical results to a higher peak, remarkable progress has been made in the performance race of machine learning applications in the past years. Yet deep learning based on artificial neural networks still remains difficult to understand as it is considered as a black-box approach. A lack of understanding of deep learning networks from the theoretical perspective would not only hinder the employment of them in applications where high-stakes decisions need to be made, but also limit their future development where artificial intelligence is expected to be robust, predictable and trustable. This paper aims to provide a theoretical methodology to investigate and train deep convolutional neural networks so as to ensure convergence. A mathematical model based on matrix representations for convolutional neural networks is first formulated and an analytic layer-wise learning framework for convolutional neural networks is then proposed and tested on several common benchmarking image datasets. The case studies show a reasonable trade-off between accuracy and analytic learning, and also highlight the potential of employing the proposed layer-wise learning method in finding the appropriate number of layers in actual implementations. |
author2 |
School of Electrical and Electronic Engineering |
author_facet |
School of Electrical and Electronic Engineering Nguyen, Huu-Thiet Li, Sitan Cheah, Chien Chern |
format |
Article |
author |
Nguyen, Huu-Thiet Li, Sitan Cheah, Chien Chern |
author_sort |
Nguyen, Huu-Thiet |
title |
A layer-wise theoretical framework for deep learning of convolutional neural networks |
title_short |
A layer-wise theoretical framework for deep learning of convolutional neural networks |
title_full |
A layer-wise theoretical framework for deep learning of convolutional neural networks |
title_fullStr |
A layer-wise theoretical framework for deep learning of convolutional neural networks |
title_full_unstemmed |
A layer-wise theoretical framework for deep learning of convolutional neural networks |
title_sort |
layer-wise theoretical framework for deep learning of convolutional neural networks |
publishDate |
2022 |
url |
https://hdl.handle.net/10356/155225 |
_version_ |
1725985659009630208 |