An overview of effect of activation functions on training and performance of artificial neural network modelling

• Main Authors: Md Munirul, Hasan, Rahman, Md Mustafizur, Suraya, Abu Bakar, Islam, Mir Sujaul, Kabir, Muhammad Nomani
• Format: Conference or Workshop Item
• Language: English
• Published: 2022
• Subjects: QA75 Electronic computers. Computer science; QA76 Computer software; T Technology (General); TA Engineering (General). Civil engineering (General); TJ Mechanical engineering and machinery
• Online Access: http://umpir.ump.edu.my/id/eprint/36927/1/An%20overview%20of%20effect%20of%20activation%20functions%20on%20training%20and%20performance%20of%20artificial%20neural%20network%20modelling.pdf; http://umpir.ump.edu.my/id/eprint/36927/; https://ncon-pgr.ump.edu.my/index.php/en/?option=com_fileman&view=file&routed=1&name=E-BOOK%20NCON%202022%20.pdf&folder=E-BOOK%20NCON%202022&container=fileman-files
• Institution: Universiti Malaysia Pahang

This paper presents an overview of the effect of the activation functions on the training and performance of artificial neural network modelling. An artificial neural network's activation functions are mathematical formulas that are essential to its design. Activation functions are a critical component of artificial neural networks since they impact the performance of the ANN model to a considerable extent. It is a function that is utilized in order to obtain the output of the node. In an artificial neural network, defining an activation function is critical, as it directly affects the network's success rate. A concise summary of some of the most frequent activation functions that are utilized in neural networks. Activation functions are defined, their properties are compared, and their advantages and disadvantages are described in this paper. This review is provided with the definitions, features, performance comparisons, merits and demerits and applications of activation function in various areas. The activation function has an impact on the development of ANN models. It is found that sigmoid, Tanh and ReLU are the most used activation function and give better performance compared to others.