An analytic end-to-end collaborative learning algorithm
In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box a...
Saved in:
Main Authors: | , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-173152 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1731522024-03-22T15:40:03Z An analytic end-to-end collaborative learning algorithm Li, Sitan Cheah, Chien Chern School of Electrical and Electronic Engineering 2023 62nd IEEE Conference on Decision and Control (CDC) Engineering End-to-End Convergence analysis In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm. Ministry of Education (MOE) Submitted/Accepted version This work was supported by the Ministry of Education (MOE) Singapore, Academic Research Fund (AcRF) Tier 1, under Grant RG65/22. 2024-03-22T01:13:48Z 2024-03-22T01:13:48Z 2024 Conference Paper Li, S. & Cheah, C. C. (2024). An analytic end-to-end collaborative learning algorithm. 2023 62nd IEEE Conference on Decision and Control (CDC). 979-8-3503-0124-3 https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding en RG65/22 © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online under IEEE Control Systems Letters at http://doi.org/10.1109/LCSYS.2023.3292034. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering End-to-End Convergence analysis |
spellingShingle |
Engineering End-to-End Convergence analysis Li, Sitan Cheah, Chien Chern An analytic end-to-end collaborative learning algorithm |
description |
In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm. |
author2 |
School of Electrical and Electronic Engineering |
author_facet |
School of Electrical and Electronic Engineering Li, Sitan Cheah, Chien Chern |
format |
Conference or Workshop Item |
author |
Li, Sitan Cheah, Chien Chern |
author_sort |
Li, Sitan |
title |
An analytic end-to-end collaborative learning algorithm |
title_short |
An analytic end-to-end collaborative learning algorithm |
title_full |
An analytic end-to-end collaborative learning algorithm |
title_fullStr |
An analytic end-to-end collaborative learning algorithm |
title_full_unstemmed |
An analytic end-to-end collaborative learning algorithm |
title_sort |
analytic end-to-end collaborative learning algorithm |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding |
_version_ |
1794549496874532864 |