Entropy learning in neural network
In this paper, entropy term is used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence entropy approach is proposed to dampen the early creation of such nodes. The entropy...
Saved in:
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
ASEAN Committee on Science and Technology
2003
|
Subjects: | |
Online Access: | http://irep.iium.edu.my/38199/1/ENTROPY_LEARNING_IN_NEURAL_NETWORK.pdf http://irep.iium.edu.my/38199/ http://astnet.asean.org/index.php?name=Main&file=content&cid=32 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Universiti Islam Antarabangsa Malaysia |
Language: | English |
id |
my.iium.irep.38199 |
---|---|
record_format |
dspace |
spelling |
my.iium.irep.381992014-09-12T01:42:54Z http://irep.iium.edu.my/38199/ Entropy learning in neural network Geok, See Ng Shi, Daming Abdul Rahman, Abdul Wahab Singh, H. T Technology (General) In this paper, entropy term is used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence entropy approach is proposed to dampen the early creation of such nodes. The entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be eliminated to reduce the memory requirements of the neural network. ASEAN Committee on Science and Technology 2003 Article REM application/pdf en http://irep.iium.edu.my/38199/1/ENTROPY_LEARNING_IN_NEURAL_NETWORK.pdf Geok, See Ng and Shi, Daming and Abdul Rahman, Abdul Wahab and Singh, H. (2003) Entropy learning in neural network. ASEAN Journal for Science and Technology Development, 20 (3&4). pp. 307-322. ISSN 0217-5460 (P), 0976-3376 (O) http://astnet.asean.org/index.php?name=Main&file=content&cid=32 |
institution |
Universiti Islam Antarabangsa Malaysia |
building |
IIUM Library |
collection |
Institutional Repository |
continent |
Asia |
country |
Malaysia |
content_provider |
International Islamic University Malaysia |
content_source |
IIUM Repository (IREP) |
url_provider |
http://irep.iium.edu.my/ |
language |
English |
topic |
T Technology (General) |
spellingShingle |
T Technology (General) Geok, See Ng Shi, Daming Abdul Rahman, Abdul Wahab Singh, H. Entropy learning in neural network |
description |
In this paper, entropy term is used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence entropy approach is proposed to dampen the early creation of such nodes. The entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be eliminated to reduce the memory requirements of the neural network. |
format |
Article |
author |
Geok, See Ng Shi, Daming Abdul Rahman, Abdul Wahab Singh, H. |
author_facet |
Geok, See Ng Shi, Daming Abdul Rahman, Abdul Wahab Singh, H. |
author_sort |
Geok, See Ng |
title |
Entropy learning in neural network |
title_short |
Entropy learning in neural network |
title_full |
Entropy learning in neural network |
title_fullStr |
Entropy learning in neural network |
title_full_unstemmed |
Entropy learning in neural network |
title_sort |
entropy learning in neural network |
publisher |
ASEAN Committee on Science and Technology |
publishDate |
2003 |
url |
http://irep.iium.edu.my/38199/1/ENTROPY_LEARNING_IN_NEURAL_NETWORK.pdf http://irep.iium.edu.my/38199/ http://astnet.asean.org/index.php?name=Main&file=content&cid=32 |
_version_ |
1643611370037444608 |