Stacked autoencoder based deep random vector functional link neural network for classification

Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input–output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsuper...

Full description

Saved in:
Bibliographic Details
Main Authors: Katuwal, Rakesh, Suganthan, Ponnuthurai Nagaratnam
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/139678
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-139678
record_format dspace
spelling sg-ntu-dr.10356-1396782021-02-03T07:22:24Z Stacked autoencoder based deep random vector functional link neural network for classification Katuwal, Rakesh Suganthan, Ponnuthurai Nagaratnam School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Random Vector Functional Link (RVFL) Deep RVFL Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input–output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks. Accepted version 2020-05-21T02:09:24Z 2020-05-21T02:09:24Z 2019 Journal Article Katuwal, R., & Suganthan, P. N. (2019). Stacked autoencoder based deep random vector functional link neural network for classification. Applied Soft Computing, 85, 105854-. doi:10.1016/j.asoc.2019.105854 1568-4946 https://hdl.handle.net/10356/139678 10.1016/j.asoc.2019.105854 2-s2.0-85074525137 85 en Applied Soft Computing © 2019 Elsevier B.V. All rights reserved. This paper was published in Applied Soft Computing and is made available with permission of Elsevier B.V. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
Random Vector Functional Link (RVFL)
Deep RVFL
spellingShingle Engineering::Electrical and electronic engineering
Random Vector Functional Link (RVFL)
Deep RVFL
Katuwal, Rakesh
Suganthan, Ponnuthurai Nagaratnam
Stacked autoencoder based deep random vector functional link neural network for classification
description Extreme learning machine (ELM), which can be viewed as a variant of Random Vector Functional Link (RVFL) network without the input–output direct connections, has been extensively used to create multi-layer (deep) neural networks. Such networks employ randomization based autoencoders (AE) for unsupervised feature extraction followed by an ELM classifier for final decision making. Each randomization based AE acts as an independent feature extractor and a deep network is obtained by stacking several such AEs. Inspired by the better performance of RVFL over ELM, in this paper, we propose several deep RVFL variants by utilizing the framework of stacked autoencoders. Specifically, we introduce direct connections (feature reuse) from preceding layers to the fore layers of the network as in the original RVFL network. Such connections help to regularize the randomization and also reduce the model complexity. Furthermore, we also introduce denoising criterion, recovering clean inputs from their corrupted versions, in the autoencoders to achieve better higher level representations than the ordinary autoencoders. Extensive experiments on several classification datasets show that our proposed deep networks achieve overall better and faster generalization than the other relevant state-of-the-art deep neural networks.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Katuwal, Rakesh
Suganthan, Ponnuthurai Nagaratnam
format Article
author Katuwal, Rakesh
Suganthan, Ponnuthurai Nagaratnam
author_sort Katuwal, Rakesh
title Stacked autoencoder based deep random vector functional link neural network for classification
title_short Stacked autoencoder based deep random vector functional link neural network for classification
title_full Stacked autoencoder based deep random vector functional link neural network for classification
title_fullStr Stacked autoencoder based deep random vector functional link neural network for classification
title_full_unstemmed Stacked autoencoder based deep random vector functional link neural network for classification
title_sort stacked autoencoder based deep random vector functional link neural network for classification
publishDate 2020
url https://hdl.handle.net/10356/139678
_version_ 1692012903850311680