Random vector functional link neural network based deep learning for regression

The deep RVFLs are inspired by the principles of the Random Vector Functional Link (RVFL) neural network. Like RVFL, the weights of the hidden layers of the deep RVFLs (dRVFL and edRVFL) are stochastically generated within a suitable range and kept constant throughout the training. In this paper, we...

Full description

Saved in:
Bibliographic Details
Main Author: Chion, Jet Herng
Other Authors: Ponnuthurai Nagaratnam Suganthan
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2020
Subjects:
Online Access:https://hdl.handle.net/10356/140217
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
Description
Summary:The deep RVFLs are inspired by the principles of the Random Vector Functional Link (RVFL) neural network. Like RVFL, the weights of the hidden layers of the deep RVFLs (dRVFL and edRVFL) are stochastically generated within a suitable range and kept constant throughout the training. In this paper, we test and evaluate the performances of the recently proposed deep RVFLs neural networks on regression problems. Through the comprehensive evaluation on 29 different UCI datasets, we show that the performances of both dRVFL and edRVFL are significantly better than the RVFL variant (OPE-RVFL) in [1] and on par with backpropagation based Deep Neural Network. Furthermore, we identify Sigmoid as the most suitable activation function for regression tasks. Finally, we propose three new deep RVFLs variants (Deep Boosted dRVFL, edRVFL+ and Two-stage edRVFL+R) which show significant improvement over the vanilla deep RVFLs. Two-stage edRVFL+R is identified as the best performing algorithm among the three newly proposed variants as it performs better than the deep neural network in 22 out of the 29 UCI datasets. The robustness of the Two-stage edRVFL+R is further confirmed by the fact that it outperforms another method (Parallel metaheuristic-ensemble of heterogeneous feedforward neural networks ) proposed in [2] by a large margin in 11 out of 14 selected datasets.