Adaptive learning rate for neural network

The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose o...

Full description

Saved in:
Bibliographic Details
Main Author: Teo, Chee Seong
Other Authors: Chua Chek Beng
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2021
Subjects:
Online Access:https://hdl.handle.net/10356/148493
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-148493
record_format dspace
spelling sg-ntu-dr.10356-1484932023-02-28T23:15:13Z Adaptive learning rate for neural network Teo, Chee Seong Chua Chek Beng School of Physical and Mathematical Sciences CBChua@ntu.edu.sg Science::Mathematics The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose of this thesis is to present the Armijo learning rate (LR) to eliminate the need of manually selecting and tuning the learning rate. We first introduce related information to our work, including the foundation of the neural network. We discuss some current methods on selecting learning rate and propose the Armijo LR. We evaluate the Armijo LR with the current methods and evaluate their performance on some image classification data sets. Bachelor of Science in Mathematical Sciences and Economics 2021-04-28T02:37:35Z 2021-04-28T02:37:35Z 2021 Final Year Project (FYP) Teo, C. S. (2021). Adaptive learning rate for neural network. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/148493 https://hdl.handle.net/10356/148493 en application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Science::Mathematics
spellingShingle Science::Mathematics
Teo, Chee Seong
Adaptive learning rate for neural network
description The learning rate is one of the most important hyper-parameters to tune in a neural network and Deep Learning. The right choice of learning rate results in a better model and faster convergence during the learning process. Time is often wasted on selecting and tuning the learning rate. The purpose of this thesis is to present the Armijo learning rate (LR) to eliminate the need of manually selecting and tuning the learning rate. We first introduce related information to our work, including the foundation of the neural network. We discuss some current methods on selecting learning rate and propose the Armijo LR. We evaluate the Armijo LR with the current methods and evaluate their performance on some image classification data sets.
author2 Chua Chek Beng
author_facet Chua Chek Beng
Teo, Chee Seong
format Final Year Project
author Teo, Chee Seong
author_sort Teo, Chee Seong
title Adaptive learning rate for neural network
title_short Adaptive learning rate for neural network
title_full Adaptive learning rate for neural network
title_fullStr Adaptive learning rate for neural network
title_full_unstemmed Adaptive learning rate for neural network
title_sort adaptive learning rate for neural network
publisher Nanyang Technological University
publishDate 2021
url https://hdl.handle.net/10356/148493
_version_ 1759855748908580864