Global convergence of online BP training with dynamic learning rate
The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on th...
Saved in:
Main Authors: | , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2013
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/99834 http://hdl.handle.net/10220/13532 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-99834 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-998342020-03-07T14:00:31Z Global convergence of online BP training with dynamic learning rate Zhang, Rui Xu, Zong-Ben Huang, Guang-Bin Wang, Dianhui School of Electrical and Electronic Engineering DRNTU::Engineering::Electrical and electronic engineering The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on the estimate of the minimum error. The global convergence theory of the online BP training procedure with the proposed learning rate is further studied. It is proved that: 1) the error sequence converges to the global minimum error; and 2) the weight sequence converges to a fixed point at which the error function attains its global minimum. The obtained global convergence theory underlies the successful applications of the online BP training procedure. Illustrative examples are provided to support the theoretical analysis. 2013-09-19T07:27:22Z 2019-12-06T20:12:09Z 2013-09-19T07:27:22Z 2019-12-06T20:12:09Z 2012 2012 Journal Article Zhang, R., Xu, Z. B., Huang, G. B., & Wang, D. (2012). Global convergence of online BP training with dynamic learning rate. IEEE transactions on neural networks and learning systems, 23(2), 330-341. 2162-237X https://hdl.handle.net/10356/99834 http://hdl.handle.net/10220/13532 10.1109/TNNLS.2011.2178315 en IEEE transactions on neural networks and learning systems © 2012 IEEE |
institution |
Nanyang Technological University |
building |
NTU Library |
country |
Singapore |
collection |
DR-NTU |
language |
English |
topic |
DRNTU::Engineering::Electrical and electronic engineering |
spellingShingle |
DRNTU::Engineering::Electrical and electronic engineering Zhang, Rui Xu, Zong-Ben Huang, Guang-Bin Wang, Dianhui Global convergence of online BP training with dynamic learning rate |
description |
The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on the estimate of the minimum error. The global convergence theory of the online BP training procedure with the proposed learning rate is further studied. It is proved that: 1) the error sequence converges to the global minimum error; and 2) the weight sequence converges to a fixed point at which the error function attains its global minimum. The obtained global convergence theory underlies the successful applications of the online BP training procedure. Illustrative examples are provided to support the theoretical analysis. |
author2 |
School of Electrical and Electronic Engineering |
author_facet |
School of Electrical and Electronic Engineering Zhang, Rui Xu, Zong-Ben Huang, Guang-Bin Wang, Dianhui |
format |
Article |
author |
Zhang, Rui Xu, Zong-Ben Huang, Guang-Bin Wang, Dianhui |
author_sort |
Zhang, Rui |
title |
Global convergence of online BP training with dynamic learning rate |
title_short |
Global convergence of online BP training with dynamic learning rate |
title_full |
Global convergence of online BP training with dynamic learning rate |
title_fullStr |
Global convergence of online BP training with dynamic learning rate |
title_full_unstemmed |
Global convergence of online BP training with dynamic learning rate |
title_sort |
global convergence of online bp training with dynamic learning rate |
publishDate |
2013 |
url |
https://hdl.handle.net/10356/99834 http://hdl.handle.net/10220/13532 |
_version_ |
1681048347464433664 |