Ensemble deep random vector functional link neural network for regression

Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the...

Full description

Saved in:
Bibliographic Details
Main Authors: Hu, Minghui, Chion, Jet Herng, Suganthan, Ponnuthurai Nagaratnam, Katuwal, Rakesh Kumar
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/165020
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-165020
record_format dspace
spelling sg-ntu-dr.10356-1650202023-03-10T15:40:16Z Ensemble deep random vector functional link neural network for regression Hu, Minghui Chion, Jet Herng Suganthan, Ponnuthurai Nagaratnam Katuwal, Rakesh Kumar School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Ensemble Neural Networks Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the performances of the dRVFLs on regression problems. Subsequently, we propose a novel regularization method boosted factor (BF), two dRVFLs variants edRVFL with skip connection (edRVFL-SC) and edRVFL with random skip connections (edRVFL-RSC) and one strategy ensemble skip connection edRVFL (esc-edRVFL) which show significant improvement over the original dRVFL. The BF is a newly introduced hyperparameter to scale the values of the activated hidden neurons to accommodate the diversity of the data, and it is also able to filter the neurons. edRVFL-SC and edRVFL-RSC are the edRVFL variants with skip connections. In edRVFL-SC, we apply dense skip connections to the edRVFL, which is inspired by the residual architecture in the deep learning area. However, due to the specificity of randomized networks, the simple skip connections are probably leading to the reuse of useless features. To address this problem, we propose a random skip connection-based edRVFL, which can keep the diversity in the latent space. esc-RVFL is an ensemble scheme that utilizes several edRVFL-RSC models trained on the different folds of the training dataset. The esc-edRVFL is identified as the best-performing algorithm through a comprehensive evaluation of 31 UCI datasets. Published version Open Access funding provided by the Qatar National Library. 2023-03-08T01:10:56Z 2023-03-08T01:10:56Z 2022 Journal Article Hu, M., Chion, J. H., Suganthan, P. N. & Katuwal, R. K. (2022). Ensemble deep random vector functional link neural network for regression. IEEE Transactions On Systems, Man, and Cybernetics: Systems, 1-12. https://dx.doi.org/10.1109/TSMC.2022.3213628 2168-2216 https://hdl.handle.net/10356/165020 10.1109/TSMC.2022.3213628 2-s2.0-85141612645 1 12 en IEEE Transactions on Systems, Man, and Cybernetics: Systems © 2022 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
Ensemble
Neural Networks
spellingShingle Engineering::Electrical and electronic engineering
Ensemble
Neural Networks
Hu, Minghui
Chion, Jet Herng
Suganthan, Ponnuthurai Nagaratnam
Katuwal, Rakesh Kumar
Ensemble deep random vector functional link neural network for regression
description Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the performances of the dRVFLs on regression problems. Subsequently, we propose a novel regularization method boosted factor (BF), two dRVFLs variants edRVFL with skip connection (edRVFL-SC) and edRVFL with random skip connections (edRVFL-RSC) and one strategy ensemble skip connection edRVFL (esc-edRVFL) which show significant improvement over the original dRVFL. The BF is a newly introduced hyperparameter to scale the values of the activated hidden neurons to accommodate the diversity of the data, and it is also able to filter the neurons. edRVFL-SC and edRVFL-RSC are the edRVFL variants with skip connections. In edRVFL-SC, we apply dense skip connections to the edRVFL, which is inspired by the residual architecture in the deep learning area. However, due to the specificity of randomized networks, the simple skip connections are probably leading to the reuse of useless features. To address this problem, we propose a random skip connection-based edRVFL, which can keep the diversity in the latent space. esc-RVFL is an ensemble scheme that utilizes several edRVFL-RSC models trained on the different folds of the training dataset. The esc-edRVFL is identified as the best-performing algorithm through a comprehensive evaluation of 31 UCI datasets.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Hu, Minghui
Chion, Jet Herng
Suganthan, Ponnuthurai Nagaratnam
Katuwal, Rakesh Kumar
format Article
author Hu, Minghui
Chion, Jet Herng
Suganthan, Ponnuthurai Nagaratnam
Katuwal, Rakesh Kumar
author_sort Hu, Minghui
title Ensemble deep random vector functional link neural network for regression
title_short Ensemble deep random vector functional link neural network for regression
title_full Ensemble deep random vector functional link neural network for regression
title_fullStr Ensemble deep random vector functional link neural network for regression
title_full_unstemmed Ensemble deep random vector functional link neural network for regression
title_sort ensemble deep random vector functional link neural network for regression
publishDate 2023
url https://hdl.handle.net/10356/165020
_version_ 1761781201548345344