PROPOSED METHODOLOGY FOR OPTIMIZING THE TRAINING PARAMETERS OF A MULTILAYER FEEDFORWARD ARTIFICIAL NEURAL NETWORKS USING A GENETIC ALGORITHM

• Main Author: AHMED ABDALLA, OSMAN
• Format: Thesis
• Language: English
• Published: 2011
• Online Access: http://utpedia.utp.edu.my/2846/1/my_thesis_osman30-1-2001...last7.pdf; http://utpedia.utp.edu.my/2846/
• Institution: Universiti Teknologi Petronas

An artificial neural network (ANN), or shortly "neural network" (NN), is a powerful mathematical or computational model that is inspired by the structure and/or functional characteristics of biological neural networks. Despite the fact that ANN has been developing rapidly for many years, there are still some challenges concerning the development of an ANN model that performs effectively for the problem at hand. ANN can be categorized into three main types: single layer, recurrent network and multilayer feed-forward network. In multilayer feed-forward ANN, the actual performance is highly dependent on the selection of architecture and training parameters. However, a systematic method for optimizing these parameters is still an active research area. This work focuses on multilayer feed-forward ANNs due to their generalization capability, simplicity from the viewpoint of structure, and ease of mathematical analysis. Even though, several rules for the optimization of multilayer feed-forward ANN parameters are available in the literature, most networks are still calibrated via a trial-and-error procedure, which depends mainly on the type of problem, and past experience and intuition of the expert. To overcome these limitations, there have been attempts to use genetic algorithm (GA) to optimize some of these parameters. However most, if not all, of the existing approaches are focused partially on the part of architecture and training parameters. On the contrary, the GAANN approach presented here has covered most aspects of multilayer feed-forward ANN in a more comprehensive way. This research focuses on the use of binaryencoded genetic algorithm (GA) to implement efficient search strategies for the optimal architecture and training parameters of a multilayer feed-forward ANN. Particularly, GA is utilized to determine the optimal number of hidden layers, number of neurons in each hidden layer, type of training algorithm, type of activation function of hidden and output neurons, initial weight, learning rate, momentum term, and epoch size of a multilayer feed-forward ANN. In this thesis, the approach has been analyzed and algorithms that simulate the new approach have been mapped out. The approach has been tested in three actual operations, in addition to standard XOR problem; where the results have shown the applicability of the proposed approach in those applications. The proposed method is considered novel as it has proven that GA-based method can be comprehensively utilized to determine multilayer feedforward ANN architecture and training parameters. This method is more effective and gives a more precise performance than existing approaches, in addition to being less human dependent. It also has a better generalization capability and training stability. In summary, the main contributions of this research are: demonstrates the strength of genetic algorithm (GA), auto designing of multilayer feed-forward ANN, and demonstrates the hybridization capability of GA with multilayer feed-forward ANN.