Morphological learning for spiking neurons with nonlinear active dendrites

• Main Author: Shaista Hussain
• Other Authors: Arindam Basu
• Format: Theses and Dissertations
• Language: English
• Published: 2016
• Subjects: DRNTU::Engineering::Electrical and electronic engineering; DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; DRNTU::Engineering::Computer science and engineering::Computing methodologies::Pattern recognition
• Online Access: https://hdl.handle.net/10356/69184
• Institution: Nanyang Technological University

There has been a lack of progress in developing spiking neuron models for pattern classification, which can achieve similar performance as state-of-the-art. To pursue this goal of creating powerful spike-based classifiers, the role of dendrites in neuronal information processing is considered. The neurobiological evidence for dendritic processing has been established in the last few years by neuroscientists across the globe. However, computational models of spiking neurons in machine learning systems have not utilized this mechanism yet. Our work attempts to bridge this gap and explore the possible computational benefits of passive delay and active ionic dendritic mechanisms. A spike-based model for pattern classification is presented which employs neurons with functionally distinct multicompartment dendritic branches. In this model, synaptic integration involves location-dependent processing of inputs on each dendritic compartment, followed by nonlinear processing of the total synaptic input on a dendrite and finally linear integration of the total dendritic output at the soma. This gives the neuron a capacity to perform a large number of input-output mappings. Firstly, a spiking neuron model is developed based on modifying delays associated with the spikes arriving at an afferent. The application of this model is demonstrated on memorizing spatio-temporal patterns by updating only a few delays corresponding to the most synchronous part of a spike pattern. This model explores the time-based computing approach to design a novel learning algorithm which provides an alternative to the traditional weight-based learning and offers the advantage of simpler hardware implementation without multipliers or digital-analog converters (DACs). The classification accuracy of the system with a load (number of patterns relative to the number of synapses) of up to 2 was shown to be about 80−100%. In our pursuit of achieving improved performance and a hardware-friendly learning algorithm, a model is further proposed which consists of nonlinear dendrites and is inspired by the structural plasticity involving correlation-based grouping of synaptic contacts onto dendrites. The model utilizes sparse synaptic connectivity, where each synapse takes a binary value, and learns the optimal input connections of a neuron. The modification of connectivity matrix renders the model suitable for implementation in address-event representation (AER) based neuromorphic systems. A modified margin-based model is also presented, which is shown to result in significant improvement in generalization performance. This performance is found to be comparable with that of standard methods like support vector machine (SVM) and extreme learning machine (ELM) on benchmark data sets and moreover, it is achieved by utilizing 10−50% less number of low resolution synapses compared to these algorithms. The structural learning rule for nonlinear dendrites is also extended to perform multiclass classification and demonstrate the application of this multiclass model on the classification of handwritten digits from the MNIST dataset. For this application, our model is shown to attain comparable performance (≈ 2% more error) with other spike-based classifiers while using much less synaptic resources, up to 10% of that used by other methods. Two approaches are used to train the model: 1) a branch-specific spike-time-based structural learning rule and; 2) a rate-based version of this rule to reduce training time. The correspondence between these two forms of learning is also established. Enhancements are proposed in this model by developing an adaptive structural scheme to learn the number of dendrites by progressively adding dendrites to the network and simultaneously forming synaptic connections on these dendrites, thereby allocating synaptic resources as per the complexity of the given problem. The model is further enhanced by using an ensemble method which combines several dendritic classifiers to achieve enhanced generalization over individual classifiers. The performance of these enhanced learning algorithms is again demonstrated on the classification of MNIST data, which shows that an ensemble model created with adaptively learned classifiers can attain accuracy of 96.2% which is at par with the best performance reported for any spiking neural network. Moreover, the ensemble has the advantage of using much less number of synapses, about 20% of other spike algorithms. Finally, the performance benefits of combined learning of delays associated with afferents and connection pattern of these afferents are investigated. This novel learning algorithm is used by a spiking neuron model comprising nonlinear dendrites with multiple delay compartments on each dendrite to classify spatio-temporal patterns. The multicompartment dendritic neurons are different from our earlier model using lumped dendritic nonlinearity in which all synaptic inputs on a dendrite were lumped together in one compartment. The combined learning rule inspired by the Tempotron rule uses correlation computations to form synaptic connections on specific delay compartments of the nonlinear dendrites. Our neurobiologically realistic multicompartment dendritic model achieves enhanced classification accuracy, which is about 5% higher than that attained by lumped dendrite scheme. Moreover, the Tempotron rule with weights quantized to 4-bits attains about 5% less accuracy than our multicompartment model, which uses binary weights, thereby rendering our proposed learning scheme more appealing for hardware implementation.