Adaptive learning rate for neural network
The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose o...
Saved in:
Main Author: | |
---|---|
Other Authors: | |
Format: | Final Year Project |
Language: | English |
Published: |
Nanyang Technological University
2021
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/148493 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-148493 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1484932023-02-28T23:15:13Z Adaptive learning rate for neural network Teo, Chee Seong Chua Chek Beng School of Physical and Mathematical Sciences CBChua@ntu.edu.sg Science::Mathematics The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose of this thesis is to present the Armijo learning rate (LR) to eliminate the need of manually selecting and tuning the learning rate. We first introduce related information to our work, including the foundation of the neural network. We discuss some current methods on selecting learning rate and propose the Armijo LR. We evaluate the Armijo LR with the current methods and evaluate their performance on some image classification data sets. Bachelor of Science in Mathematical Sciences and Economics 2021-04-28T02:37:35Z 2021-04-28T02:37:35Z 2021 Final Year Project (FYP) Teo, C. S. (2021). Adaptive learning rate for neural network. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/148493 https://hdl.handle.net/10356/148493 en application/pdf Nanyang Technological University |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Science::Mathematics |
spellingShingle |
Science::Mathematics Teo, Chee Seong Adaptive learning rate for neural network |
description |
The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose of this thesis is to present the Armijo learning rate (LR) to eliminate the need of manually selecting and tuning the learning rate. We first introduce related information to our work, including the foundation of the neural network. We discuss some current methods on selecting learning rate and propose the Armijo LR. We evaluate the Armijo LR with the current methods and evaluate their performance on some image classification data sets. |
author2 |
Chua Chek Beng |
author_facet |
Chua Chek Beng Teo, Chee Seong |
format |
Final Year Project |
author |
Teo, Chee Seong |
author_sort |
Teo, Chee Seong |
title |
Adaptive learning rate for neural network |
title_short |
Adaptive learning rate for neural network |
title_full |
Adaptive learning rate for neural network |
title_fullStr |
Adaptive learning rate for neural network |
title_full_unstemmed |
Adaptive learning rate for neural network |
title_sort |
adaptive learning rate for neural network |
publisher |
Nanyang Technological University |
publishDate |
2021 |
url |
https://hdl.handle.net/10356/148493 |
_version_ |
1759855748908580864 |