Enhanced extreme learning machine for general regression and classification tasks

• Main Author: Mahmood, Saif F
• Format: Thesis
• Language: English
• Published: 2020
• Subjects: Artificial intelligence - Engineering applications; Machine learning; Regression analysis
• Online Access: http://psasir.upm.edu.my/id/eprint/92877/1/FK%202020%2097%20-%20IR.1.pdf; http://psasir.upm.edu.my/id/eprint/92877/
• Institution: Universiti Putra Malaysia

Extreme Learning Machine (ELM) is a single hidden layer feedforward neural network which randomly chooses hidden nodes and analytically determines the output weights using least square method. Despite its popularity, ELM has a number of challenges worth to investigating for improving the usability of ELM in a more advanced application. This thesis focusses on challenges namely design architecture and learning technique. The first challenge is to select the optimal number of hidden nodes for ELM in different application. To address this problem, a new approach referred to SVM-ELM is proposed, which utilizes 1-norm support vector machine (SVM) to the hidden layer matrix of ELM in order to automatically discover the optimal number of hidden nodes. The method is developed for regression task by using mean/ median of ELM training errors which is then used as threshold for separating the training data and converting the continuous targets to binary. This will allow projection to 1- norm SVM dimension in order to find the best number of nodes as support vectors. Second problem in ELM, is the restriction in performance of ELM in terms of training time and model generalization, due to the complexity of singular value decomposition (SVD) for computing the Moore-Penrose generalized inverse of the hidden layer matrix, especially on a large matrix. To address this issue, a fast adaptive shrinkage/thresholding algorithm ELM (FASTA-ELM) which uses an extension of forward-backward splitting (FBS) to compute the smallest norm of the output weights in ELM is presented. The proposed FASTA-ELM replaces the analytical step usually solved by SVD with an approximate solution through proximal gradient method, which dramatically speeds up the training time and improves the generalization ability in classification task. The performance of FASTA-ELM is evaluated on face gender recognition problem and the result is comparable to other state-of-theart methods, with significantly reduced training time. For instance, the training time of 1000 nodes ELM is 18.11 s, while FASTA-ELM completed in 1.671 s. The proposed modification to the ELM shows significant improvement to the conventional ELM in terms of training time and accuracy, and provide good generalization performance in regression and classification tasks.